Methods for performing DAF data filtering and padding

ABSTRACT

A method for padding, filtering, denoising, image enhancing and increased time-frequency acquisition is described for digitized data of a data set where unknown data is estimated using real data by adding unknown data points in a manner that the padding routine can estimate the interior data set including known and unknown data to a given accuracy on the known data points. The method also provides filtering using non-interpolating, well-tempered distributed approximating functional (NIDAF)-low-band-pass filters. The method also provides for symmetric and/or anti-symmetric extension of the data set so that the data set may be better refined and can be filtered by Fourier and other type of low frequency or harmonic filters.

RELATED APPLICATIONS

This application claims provisional priority to U.S. ProvisionalApplication Ser. No. 60/077,860, filed Mar. 13, 1998.

GOVERNMENT SUPPORT

This invention was supported in whole or in part, by grant No. 1-5-51749from the National Science Foundation and grant number N-00014-K-0613from the Department of the Navy, Office of Naval Research. TheGovernment has certain rights in the invention.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to methods using distributed approximatingfunctionals (DAF), DAF-wavelets and DAF-neural networks for filtering,denoising, processing, restoring, enhancing, padding, or other signalprocessing procedures directed to images, signals, 1D, 2D, 3D . . . nDspectra, X-ray spectra, CAT scans, MRI scans, NMR, and otherapplications that require data processing at or near the theoreticallimit of resolutions.

More particularly, the present invention relates to the use ofinfinitely smooth DAFs in combination with other signal processingtechniques to provide methods and apparatuses utilizing such methodsthat can enhance image, spectral, or other signal data and decrease thetime need to acquire images, spectra or other signals.

2. Description of the Related Art

Many techniques currently exist for processing images, refining spectra,analyzing data or the like. Many of these techniques are well-known andused extensively. However, these techniques generally suffer from one ormore limitation on their ability to enhance signal or image andconstruct or restore missing or lost data, especially if the userdesires the error inherit in signal acquisition and the error introducedby the processing technique to be as small as possible, i.e., as closeas possible to Heisenberg's uncertainty principle.

Thus, there is a need in the art for improved techniques for processingacquired data whether in the form on a image, a spectra, amultidimensional spectra or the like so that the error due to processingcan be minimized which can increase resolution and decrease acquisitiontimes.

SUMMARY OF THE INVENTION

The present invention provides a method implemented on a digitalprocessing device or stored in a memory readable by a digital processingunit which uses distributed approximating functionals (DAFs) to enhanceand improve signal, image and multidimensional data constructsprocessing and to decrease acquisition time of real world spectrometrictechniques that operate on square wave type frequency domains whichrequire a large acquisition time to capture the signal from noise.Shorten the acquisition time, the more noise and less resolution anddefinition the spectra will have. The limit for simultaneously acquiringfrequency information in time is given by a variant of Heisenberg'suncertainty principle, i.e., ΔωΔt≦1. The methods of the present providesmethods for image and signal processing where the accuracy and precisionof the final signal and image closely approaches the uncertaintyprinciples maximum accuracy and precision. The methods can be made toapproach uncertainty principle accuracy via increased computationalcost, but the real power of the methods of this invention is to yieldimproved measurements at a given Δω and Δt product.

The present invention also provides methods for improving X-ray andmagnetic imaging techniques, especially mammogram images using the DAFand DAF processing techniques set forth herein.

The present invention also provides a mammogram imaging system ofconventional design, the X-ray data derived thereform is then enhancedvia DAF processing in an associated digital processing unit.

DESCRIPTION OF THE DRAWINGS

The invention can be better understood with reference to the followingdetailed description together with the appended illustrative drawings inwhich like elements are numbered the same.

DAF TREATMENT OF NOISY SIGNALS

FIG. 1(a) depicts the Hermite DAF in coordinate space and FIG. 1(b)depicts the frequency space, respectively. The solid line is for M=88,σ=3.05 and the dashed line is for M=12, ˜σ=4. The solid line is close tothe interpolation region and the dashed line is in the low passfiltering region. The frequency in FIG. 1(b) has been multiplied by afactor of the grid spacing.

FIG. 2 depicts extrapolation results for the function in Equation (15).The solid line is the exact function. The long dashed line, the shortdashed line and the plus symbols are the extrapolation results forHermite DAF parameters σ/Δ=7.6, ˜7.8, and 8.0 respectively. In ournumerical application, only the values at the grid points less than −1.2are assumed to be known.

FIG. 3 depicts sine function in Equation (17) with 50% random noiseadded to the values at even spaced grids from 0 to 219 (solid line) andthe periodically extended 36 function values (plus symbols) withσ/Δ=10.5. The exact values in the extended domain are also plotted(solid line) in this figure.

FIG. 4(a) depicts the L_(∞) error and FIG. 4(b) the signature of theperiodic padding of the noisy sine function as a function of DAFparameter σ/Δ. The M is fixed to be 6.

FIG. 5(a) depicts the L_(∞) error and FIG. 5(b) the signature of the DAFsmoothing to the periodically extended noisy sine function as a functionof σ/Δ. The M is fixed to be 12.

FIG. 6 depicts periodic extension of the nonperiodic function (withnoised added) in Equation (15). (a) The 220 known values of the functionin the range [−7,10] with 20% random noise (solid line) and the 36 exactvalues of the function(dashed line). Note that the function is notperiodic at all in the range of 256 grid points. (b) The periodicallyextended function with σ/Δ=5.2. Note the smoothness and periodicproperty of the function.

FIG. 7 depicts the periodic extension signature of the noisy function inFIG. 5 as a function of σ/Δ. The M is fixed to be 6.

FIG. 8(a) depicts the L_(∞) error and FIG. 8(b) the signature of the DAFSMOOTHING to the periodically extended noisy function in FIG. 6(b) as afunction of σ/Δ. The M is fixed to be 12 for the DAF-smoothing.

FIG. 9 depicts a comparison of DAF smoothed result to the signal in FIG.6(b) at σ/Δ=9 (solid line), and with the exact function (dashed line) inEquation (15).

GENERALIZED SYMMETRIC INTERPOLATING WAVELETS

FIGS. 10(a-b) depict πband-limited interpolating wavelets (a) Sincfunction and (b) depicts the Sinclet wavelet.

FIG. 11 depicts interpolating Cardinal Spline (D=5).

FIGS. 12(a-b) depict interpolating wavelets by auto-correlation shell(D=3) with (a) Daubechies wavelet (b) Dubuc wavelet.

FIG. 13 depicts a Lifting scheme.

FIGS. 14(a-b) depict Lagrange Wavelets with D=3 (a) Scaling function,(b) Wavelet, (c) Dual scaling function, and (d) Dual wavelet.

FIGS. 15(a-b) depict Frequency Response of Equivalent Filters (D=3) (a)Decomposition and (b) Reconstruction.

FIGS. 16(a-e) depict non-regularized Lagrange Wavelets (M=5) (a)Lagrange polynomial, (b) Scaling function, (c) Wavelet, (d) Dual scalingfunction, and (e) Dual wavelet.

FIGS. 17(a-d) depict B-Spline Lagrange DAF Wavelets (N=4, η=2) (a)Scaling function, (b) Wavelet, (c) Dual scaling function, and (d) Dualwavelet.

FIGS. 18(a-b) depict Frequency Response of Equivalent Filters (N=4, η=2)(a) Decomposition and (b) Reconstruction.

FIG. 19 depicts a Mother wavelet comparison (N=4, η=2) Solid: B-splineLagrange; dotted: Gaussian-Lagrange.

FIG. 20 depicts a Gibbs overshoot of the Sinc FIR.

FIGS. 21(a-d) depict Sinc Cutoff Wavelets (M=9) (a) Scaling, (b)Wavelet, (c) Dual scaling, and (d) Dual wavelet.

FIGS. 22(a-d) depicts B-Spline Lagrange DAF Wavelets (N=5, η=3) (a)Scaling, (b) Wavelet, (c) Dual scaling, and (d) Dual wavelet.

FIGS. 23(a-b) depicts Frequency Response of Equivalent Filters (N=5,η=3) (a) Decomposition (b) and (b) Reconstruction.

FIG. 24 depicts a Mother Wavelet Comparison (N=4, η=2) Solid: B-splineSinc; dotted: Gaussian Sinc.

FIGS. 25(a-b) depict Dubuc wavelets (D=3) (a) Inner scaling (b) Boundaryscaling.

FIGS. 26(a-b) depict Generalized Sinc wavelets (N=2, η=2) (a) Innerscaling and (b) Boundary scaling.

FIGS. 27(a-b) depict Frequency response comparison of boundary filters(a) Halfband Lagrange wavelet and (b) Sinc-DAF wavelet.

FIGS. 28(a-b) depict Target extraction from color background (a)Original pilot view and (b) DAF-wavelet restoration.

FIGS. 29(a-b) depicts a Visual Color Image Restoration (a) Noisy girland (b) Our restoration.

FIGS. 30(a-c) depicts an enhancement of Database 1 (a) OriginalMammogram, (b) Linear enhancement and (c) Non-linear enhancement.

FIGS. 31(a-b) depict an enhancement of Database 2 (a) Original mammogramand (b) Enhancement.

DUAL WINDOW SELECTIVE AVERAGING FILTER

FIGS. 32(a-c) depicts the filtering of blocked signals corrupted byGaussian noise: (a) noise-free blocks signal; (b) noisy signal(MSE=1.00, MAE=0.80); and (c) filtered signal by DWSAF (MSE=6.19E-2,MAE=6.49E-3).

FIGS. 33(a-b) depict image restorations from lower noise Lena image: (a)corrupted image (PSNR=22.17 dB) and (b) restored image by DWSAF(PSNR=30.69 dB).

FIGS. 34(a-b) depict image restorations from higher noise Lena image:(a) corrupted image (PSNR=18.82 dB) and (b) restored image by DWSAF(PSNR=28.96 dB).

LAGRANGE WAVELETS FOR SIGNAL PROCESSING

FIGS. 35(a-b) depict πband-limited interpolating wavelets (a) Sincfunction and (b) Sinclet wavelet.

FIG. 36 depicts an Interpolating Cardinal Spline (D=5).

FIGS. 37(a-b) depict Interpolating wavelets by auto-correlation shell(D=3) (a) Daubechies wavelet (b) Dubuc wavelet.

FIG. 38 depicts Lifting scheme.

FIGS. 39(a-d) depict Lagrange Wavelets with D=3 (a) Scaling, (b)Wavelet, (c) Dual scaling (d) Dual wavelet.

FIGS. 40(a-b) depict Frequency Response of Equivalent Filters (D=3) (a)Decomposition and (b) Reconstruction.

FIGS. 41(a-d) depict Lagrange Wavelets with D=9 (a) Scaling, (b)Wavelet, (c) Dual scaling, and (d) Dual wavelet.

FIGS. 42(a-b) depict Frequency Response of Equivalent Filters (D=9) (a)Decomposition and (b) Reconstruction.

FIGS. 43(a-e) depict Non-regularized Lagrange Wavelets (M=5) (a)Lagrange polynomial, (b) Scaling, (c) Wavelet, (d) Dual scaling, and (e)Dual wavelet.

FIGS. 44(a-c) depict B-Spline Lagrange DAF Wavelets (N=4, η=2) (a)Scaling, (b) Wavelet, (c) Dual scaling, (d) Dual wavelet.

FIGS. 45(a-b) depict Frequency Response of Equivalent Filters (N=4, η=2)(a) Decomposition and (b) Reconstruction.

FIG. 46 depicts Mother Wavelet Comparison (N=4, η=2) Solid: B-splineLagrange; dotted: Gaussian Lagrange.

FIGS. 47(a-b) depict Nonlinear Masking Functionals (a) Donoho Hard LogicNonlinearity and (b) Softer Logic Nonlinearity.

FIGS. 48(a-c) depict 2D Lagrange wavelets for image processing (a)Scaling, (b) Vertical, (c) Horizontal and (d) Diagonal wavelets.

FIGS. 49(a-c) depict VGN image processing for Lena (a) Noisy Lena, (b)Median filtering result (c) our method.

FIGS. 50(a-c) depict VGN processing for Barbara (a) Noisy Barbara, (b)Median filtering result, (c) Our method.

IMAGE ENHANCEMENT NORMALIZATION

FIGS. 51(a-b) depicts (a) an Original Mammogram and (b) depicts anEnhanced result.

FIGS. 52(a-b) depicts (a) an Original Mammogram and (b) depicts anEnhanced result.

VARYING WEIGHT TRIMMED MEAN FILTER FOR THE RESTORATION OF IMPULSECORRUPTED IMAGES

FIG. 53 depicts the weight function of Equation (166) for A=2;

FIGS. 54(a-f) depicts image restorations from 40% impulse noisecorrupted Lena image; (a) shows the original Lena picture; (b) showsnoise image; (c) shows median Filtering (3×3), PSNR=28.75; (d) α-TMF(3×3), PSNR=27.49; (e) VWTMF (3×3), PSNR=29.06; and (f) VWTMF switch(3×3), PSNR=31.34.

A NEW NONLINEAR IMAGE FILTERING TECHNIQUE

FIGS. 55(a-d) Image restoration from 60% impulse noise: (a) Corruptedimage, (b) Filtering by Sun and Nevou's median switch scheme, (c) Ourfiltering, and (d) Our modified filtering.

FIGS. 56(a-d) Image restorations from 40% impulse noise: (a) Corruptedimage, (b) Median filtering (3×3), (c) Median filtering (5×5), and (d)Our filtering.

BIOMEDICAL SIGNAL PROCESSING USING A NEW CLASS OF WAVELETS

FIG. 57 depicts a Hermite DAF (M=8 and σ=1).

FIG. 58(a) depicts an original mammogram.

FIG. 58(b) depicts an enhanced mammogram using the DAF of FIG. 57.

FIG. 59 depicts am ECG criterion Characteristic for Diagnosis.

FIGS. 60(a-c) depicts a ECG filtering: (a) original ECG, (b) low-passfiltering, and (c) our filtering.

NONLINEAR QUINCUNX FILTERS

FIGS. 61(a-c) depict traditional trivial windows: (a) 1×1 window (singlepixel), (b) 3×3 window, and (c) 5×5 window.

FIG. 62 depicts arbitrary quincunx extension of symmetric neighboringbasket windows.

FIG. 63 depicts quincunx basket selection for the filtering.

FIGS. 64(a-b) depict image restorations from 40% impulse noise: (a)corrupted image and (b) our filtering.

FIGS. 65(a-c) depict wavelet noise removal: (a) Gaussian-degraded Lena,(b) DAF wavelet thresholding, and (c) DAF wavelet + quincunx filtering.

VISUAL MULTIRESOLUTION COLOR IMAGE RESTORATION

FIG. 66 depicts a cube model of RGB color.

FIG. 67 depicts an alternate representation.

FIG. 68 depicts a hexagon projections of color tube.

FIGS. 69(a-c) depict test results from restoration: (a) Noisy Lena, (b)Median filtering, and (c) VGN restoration.

FIGS. 70(a-c) depict test Results from restoration: (a) Noisy girl, (b)Median filtering, and (c) VGN restoration.

MAMMOGRAM ENHANCEMENT USING GENERALIZED SINC WAVELETS

FIGS. 71(a-b) depict π band-limited interpolating wavelets: (a) Sincfunction and (b) Sinclet wavelet.

FIG. 72 depicts a Fourier Gibbs overshot of Sinc FIR implementation.

FIGS. 73(a-d) depict Sinc Cutoff Wavelets (M=9) (a) Scaling, (b)Wavelet, (c) Dual scaling, and (d) Dual wavelet.

FIGS. 74(a-d) depict B-Spline Lagrange DAF Wavelets (N=5, η=3) (a)Scaling, (b) Wavelet, (c) Dual scaling, and (d) Dual wavelet.

FIGS. 75(a-b) depict Frequency Response of Equivalent Filters (N=5,η=3): (a) Decomposition and (b) Reconstruction.

FIG. 76 depicts a Mother Wavelet Comparison (N=4, η=2) Solid: B-splineSinc; dotted: Gaussian Sinc.

FIGS. 77(a-b) depict Nonlinear Masking Functionals: (a) Donoho HardLogic Nonlinearity and (b) Softer Logic Nonlinearity.

FIGS. 78(a-b) depict Mammogram enhancement: (a) Original mammogram and(b) Multiresolution enhancement by DAF-wavelet.

DUAL PROPAGATION INVERSION OF FOURIER AND LAPLACE SIGNALS

FIG. 79 depicts the auxiliary function, {hacek over (C)}(t;α,ω₀), att=0, as a function of the frequency ω₀.

FIG. 80 depicts the truncated sine function f(ω)=sin(ω); 0≦ω≦Π, and thecalculated spectrum obtained by the dual propagation inversionprocedure. The noiseless time domain signal was sampled between−45≦t≦45. The two are visually indistinguishable.

FIG. 81 depicts with dotted line: The calculated spectrum f(ω) obtainedfrom the time signal corrupted by random noise of 20%. with solid line:The calculated spectrum obtained from the noise-free time signal. Bothclean and corrupted signals were sampled between −45≦t≦45.

FIG. 82 depicts a Cross-hatched line: the calculated spectrum ƒ_(DPI)(ω)obtained from the noise-free time domain signal, sampled between −5≦t≦5.Solid line is the original truncated sine function.

FIG. 83 depicts a Cross-hatched line: the calculated spectrum ƒ_(DPI)(ω)obtained using the DAF-padding values for 5≦|t|≦7.5, joined smoothly tothe analytical tail-function (see text). Careful comparison with FIG. 82shows a reduction of the aliasing due to signal truncation.

DISTRIBUTED APPROXIMATION FUNCTIONAL WAVELET NETS

FIG. 84 depicts a Hermite DAF (M=8 and σ=1).

FIG. 85 depicts an ECG criterion Characteristic for Diagnosis.

FIGS. 86(a-c) depicts an ECG filtering: (a) original ECG, (b) low-passfiltering; and (c) DAF wavelet net filtering.

FIGS. 87(a-c) depicts an EMG filtering: (a) original EMG, (b) low-passfiltering; and (c) DAF wavelet net filtering.

PERCEPTUAL NORMALIZED SUBBAND IMAGE RESTORATION

FIG. 88 depicts a frequency response of GLDAF equivalent filters.

FIGS. 89(a-d) depict Lagrange DAF Wavelets: (a) GLDAF, (b) GLDAFwavelet, (c) dual GLDAF and (d) dual GLDAF wavelet.

DISTRIBUTED APPROXIMATING FUNCTIONAL APPROACH TO IMAGE RESTORATION

FIGS. 90(a-c) depicts a Hermite DAF: (a) in coordinate space; (b) infrequency space; and the first order derivative in coordinate space. Thesolid line is for σ=3.54 and M=120 and the dashed line is for σ=2.36 andM=130.

FIG. 91 depicts the original Lena Image (240×240).

FIGS. 92(a-b) depict image restoration: (a) degraded Lena image,PSNR=22.14dB and (b) restored Lena image, PSNR=30.14dB.

FIGS. 93(a-b) depict image restoration: (a) degraded Lena image,PSNR=18.76dB and (b) restored Lena, image, PSNR=29.19dB.

QUINCUNX INTERPOLATION 2D AND 3D WAVELET DAFS

FIG. 94 depicts a quincunx interpolation schemes.

FIG. 95 depicts another quincunx interpolation scheme.

FIG. 96 depicts another quincunx interpolation scheme.

FIG. 97 depicts another quincunx interpolation scheme.

DETAILED DESCRIPTION OF THE INVENTION

The inventors have found that a signals, images and multidimensionalimaging data can be processed at or near the uncertainty principlelimits with DAFs and various adaptation thereof which are described inthe various section of this disclosure.

DAF TREATMENT OF NOISY SIGNALS Introduction

Experimental data sets encountered in science and engineering containnoise due to the influence of internal interferences and/or externalenvironmental conditions. Sometimes the noise must be identified andremoved in order to see the true signal clearly, to analyze it, or tomake further use of it.

Signal processing techniques are now widely applied not only in variousfields of engineering but also in physics, chemistry, biology, andmedicine. Example problems of interest include filter diagonalization,solvers for eigenvalues and eigenstates [1-3], solution of ordinary andpartial differential equations [4-5], pattern analysis,characterization, and denoising [6-7], and potential energy surfacefitting [8]. One of the most important topics in signal processing isfilter design. Frequency-selective filters are a particularly importantclass of linear, time-invariant (LTI) analyzers [9]. For a givenexperimental data set, however, some frequency selective finite impulseresponse (FIR) filters require a knowledge of the signal in both theunknown “past” and “future” domains. This is a tremendously challengingsituation when one attempts to analyze the true signal values near theboundaries of the known data set. Direct application of this kind offilter to the signal leads to aliasing; i.e., the introduction ofadditional, nonphysical frequencies to the true signal, a problem called“aliasing” [9]. Additionally, in the implementation of the fast Fouriertransform (FFT) algorithm [9], it is desirable to have the number ofdata values or samples to be a power of 2. However, this condition isoften not satisfied for a given set of experimental measurements, so onemust either delete data points or augment the data by simulating in somefashion, the unknown data.

Determining the true signal by extending the domain of experimental datais extremely difficult without additional information concerning thesignal, such as a knowledge of the boundary conditions. It is an eventougher task, using the typical, interpolation approach, when the signalcontains noise. Such interpolation formulae necessarily return datavalues that are exact on the grids; but they suffer a loss of accuracyoff the grid points or in the unknown domain, since they reproduce thenoisy signal data exactly, without discriminating between true signaland the noise. In this disclosure, an algorithm is introduced that makesuse of the well-tempered property of “distributed approximatingfunctionals” (DAFs) [10-13]. The basic idea is to introduce apseudo-signal by adding gaps at the ends of the known data, and assumingthe augmented signal to be periodic. The unknown gap data are determinedby solving linear algebraic equations that extremize a cost function.This procedure thus imposes a periodic boundary condition on theextended signal. Once periodic boundary conditions are enforced, thepseudo-signal is known everywhere and can be used for a variety ofnumerical applications. The detailed values in the gap are usually notof particular interest. The advantage of the algorithm is that theextended signal adds virtually no aliasing to the true signal, which isan important problem in signal processing. Two of the main sources ofaliasing are too small a sampling frequency and truncation of the signalduration. Another source of error is contamination of the true signal bynumerical or experimental noise. Here we are concerned only with how toavoid the truncation induced and noisy aliasing of the true signal.

Distributed approximating functions (DAFs) were recently introduced[10-11] as a means of approximating continuous functions from valuesknown only on a discrete sample of points, and of obtaining approximatelinear transformations of the function (for example, its derivatives tovarious orders). One interesting feature of a class of commonly-usedDAFs is the so-called well-tempered property [13]; it is the key to theuse of DAFs as the basis of a periodic extension algorithm. DAFs differfrom the most commonly used approaches in that there are no specialpoints in the DAF approximation; i.e., the DAF representation of afunction yields approximately the same level of accuracy for thefunction both on and off the grid points. However, we remark that theapproximation to the derivatives is not, in general quite as accurate asthe DAF approximation to the function itself because the derivatives ofL²-functions contain an increased contribution from high frequencies. Bycontrast, most other approaches yield exact results for the function onthe grid points, but often at the expense of the quality of the resultsfor the function elsewhere [13]. DAFs also provide a well-temperedapproximation to various linear transformations of the function. DAFrepresentations of derivatives to various orders will yieldapproximately similar orders of accuracy as long as the resultingderivatives remain in the DAF class. The DAF approximation to a functionand a finite number of derivatives can be made to be of machine accuracywith a proper choice of the DAF parameters. DAFs are easy to applybecause they yield integral operators for derivatives. These importantfeatures of DAFs have made them successful computational tools forsolving various linear and nonlinear partial differential equations(PDEs) [14-17], for pattern recognition and analysis [6], and forpotential energy surface fitting [8]. The well-tempered DAFs also arelow-pass filters. In this disclosure, the usefulness of DAFs as low passfilters is also studied when they are applied to a periodically extendednoisy signal. For the present purpose, we assume that the weak noise ismostly in the high frequency region and the true signal is bandwidthlimited in frequency space, and is larger than the noise in this samefrequency region. To determine when the noise is eliminated, weintroduce a signature to identify the optimum DAF parameters. Thisconcept is based on computing the root-mean-square of the smoothed datafor given DAF parameters. By examining its behavior as a function of theDAF parameters, it is possible to obtain the overall frequencydistribution of the original noisy signal. This signature helps us toperiodically extend and filter noise in our test examples.

The first example is a simple, noisy periodic signal, for which the DAFperiodic extension is a special case of extrapolation. The second is anonperiodic noisy signal. After performing the periodic extension andfiltering, it is seen that the resulting signal is closely recreates thetrue signal.

DISTRIBUTED APPROXIMATING FUNCTIONALS

DAFs can be viewed as “approximate identity kernels” used to approximatea continuous function in terms of discrete sampling on a grid [10-13].One class of DAFs that has been particularly useful is known as thewell-tempered DAFs, which provide an approximation to a function havingthe same order of accuracy both on and off the grid points. Aparticularly useful member of this class of DAFs is constructed usingHermite polynomials, and prior to discretization, is given by$\begin{matrix}{{ {{{\delta_{M}( {x - x^{\prime}} }}\sigma} ) = {{\frac{1}{\sigma}\exp} - {( \frac{( {x - x^{\prime}} )^{2}}{2\sigma^{2}} ){\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2\pi}{n!}}{H_{2n}( \frac{( {x - x^{\prime}} )}{\sqrt{2}\sigma} )}}}}}},} & (1)\end{matrix}$where σ, M are the DAF parameters and H_(2n) is the usual (even) Hermitepolynomial. The Hermite DAF is dominated by its Gaussian envelope,exp(−(x−x′)²/2σ²), which effectively determines the extent of thefunction. The continuous, analytic approximation to a function ƒ(x)generated by the Hermite DAF is $\begin{matrix}{ {{{f(x)} \approx {f_{DAF}(x)}} = {\int_{- \infty}^{\infty}{{{\delta_{M}( {x - x^{\prime}} }}\sigma}}} ){f( x^{\prime} )}\quad{{\mathbb{d}x^{\prime}}.}} & (2)\end{matrix}$

Given a discrete set of functional values on a grid, the DAFapproximation to the function at any point x (on or off the grid) can beobtained by $\begin{matrix}{{f_{DAF}(x)} = {\Delta{\sum\limits_{j}{{\delta_{M}( {x - x_{j}} \middle| \sigma )}{{f( x_{j} )}.}}}}} & (3)\end{matrix}$where Δ is the uniform grid spacing (non-uniform and even randomsampling can also be used by an appropriate extension of the theory).The summation is over all grid points (but only those close to xeffectively contribute). Similarly, for a two-dimensional functionƒ(x,y), one can write $\begin{matrix}{{ { {{f_{DAF}( {x,y} )} = {\Delta_{x}\Delta_{y}{\sum\limits_{j_{1},j_{2}}\quad{{{\delta_{M_{1}}( {x - x_{j_{1}}} }}\sigma_{1}}}}} ){\delta_{M_{2}}( {y - y_{j_{2}}} }\sigma_{2}} ){f( {x_{j_{1}},y_{j_{2}}} )}},} & (4)\end{matrix}$using a simple direct product. In FIG. 1, we plot Hermite DAFs obtainedwith two different sets of parameters, in (a) coordinate space, and (b)frequency space. The solid line (σ=3.05, M=88) is more interpolativecompared to the DAF given by the dashed line (σ=4, M=12). The latter ismore smoothing when applied to those functions whose Fourier compositionlies mostly under the σ=3.05, M=88 DAF window. This results from theσ=4, M=12 DAF window being narrower in Fourier space than that of theDAF with σ=3.05, M=88. The discretized Hermite DAF is highly banded incoordinate space due to the presence of the Gaussian envelope, whichmeans that only a relatively small number of values are needed on bothsides of the point x in Equation (3), as can be clearly seen from FIG.1(a). This is in contrast to the sinc function$\frac{\sin( {\omega\quad x} )}{\pi\quad x}.$From FIG. 1(b), we see that the Hermite DAF is also effectivelybandwidth-limited in frequency space. With a proper choice ofparameters, the Hermite DAF can produce an arbitrarily good filter (seethe dashed line in FIGS. 1(a-b). Once the boundary condition is fixedfor a data set, Equation (3) or (4) can then be used to eliminate thehigh frequency noise of that data set. As long as the frequencydistribution of the noise lies outside the DAF plateau (FIG. 1(b)), theHermite DAF will eliminate the noise regardless of its magnitude.

The approximate linear transformations of a continuous function can alsobe generated using the Hermite DAF. One particular example isderivatives of a function to various orders, given by $\begin{matrix}{{{{f^{(l)}(x)} \approx f_{DAF}^{(l)}} = {\int_{- \infty}^{\infty}{{\delta_{M}^{(l)}( {x - x^{\prime}} \middle| \sigma )}{f( x^{\prime} )}\quad{\mathbb{d}x^{\prime}}}}},} & (5)\end{matrix}$where δ_(M) ⁽¹⁾ (x−x′|σ) is the lth derivative of δ_(M) (x−x′|σ), withrespect to x, and is given explicitly by $\begin{matrix}{= {\frac{2^{{- l}/2}}{\sigma^{l + 1}}{\exp( \frac{- ( {x - x^{\prime}} )^{2}}{2\sigma^{2}} )}{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}( {- 1} )^{l}\frac{1}{\sqrt{2\pi}{n!}}{H_{{2n} + l}( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )}}}}} & (6)\end{matrix}$When uniformly discretized by quadrature, Equation (5) gives$\begin{matrix}{{{f^{(l)}(x)} \approx {f_{DAF}^{(l)}(x)}} = {\Delta{\sum\limits_{j}\quad{{\delta_{M}^{(l)}( {x - x_{j}} \middle| \sigma )}{f( x_{j} )}}}}} & (7)\end{matrix}$Expressions (5) and (7) are extremely useful in solving linear andnonlinear partial differential equations (PDEs) [14-17] because thedifferential operation has been expressed as an integration. With ajudicious choice of the DAF parameters, it is possible to providearbitrarily high accuracy for estimating the derivatives.

METHOD OF DATA EXTRAPOLATION

Case I. Filling a Gap

Suppose we have a set of uniformly spaced grid points on the infiniteline, and a continuous function, ƒ(x), known on all grid points exceptfor the set {x₁, . . . , x_(K)}. Assuming that ƒ(x) is in the DAF-class,we can estimate the unknown values by minimizing the cost function,$\begin{matrix}{C \equiv {\sum\limits_{p = {- \infty}}^{\infty}\quad{W_{p}( {{f( x_{p} )} - {f_{DAF}( x_{p} )}} )}^{2}}} & (8)\end{matrix}$where, W_(p) is a weight assigned to the point x_(p), and in thisdisclosure it is chosen to be 1 on a finite grid and 0 elsewhere;ƒ_(DAF)(x_(p)) is the DAF approximation to the function at the pointx_(p). From Equations (3) and (8), we have $\begin{matrix}{C \equiv {\sum\limits_{p = {- \infty}}^{\infty}{\quad{W_{p}( {{f( x_{p} )} - {\sum\limits_{t = {p - w}}^{p + w}\quad{{\delta_{M}( {x_{p} - x_{t}} \middle| \sigma )}{f( x_{t} )}}}} )}^{2}}}} & (9)\end{matrix}$where w is the half DAF bandwidth. We minimize this cost function withrespect to the unknown values, {ƒ(x_(J)), . . . , ƒ(x_(K))}, accordingto $\begin{matrix}{{\frac{\partial C}{\partial{f( x_{l} )}} = 0},\quad{J \leq l \leq K},} & (10)\end{matrix}$to generate the set of linear algebraic equations, $\begin{matrix}{{{\sum\limits_{p = {- \infty}}^{\infty}{\quad 2{W_{p}( {{f( x_{p} )} - {\sum\limits_{t = {p - w}}^{p + w}\quad{\delta_{M}( {x_{p} - x_{l}} }}} \middle| { {\sigma\quad} ){f( x_{t} )}} )}( {\delta_{pl} - {\delta_{M}( {x_{p} - x_{l}} \middle| \sigma )}}  \quad )}} = 0},{J \leq l \leq K},} & (11)\end{matrix}$where the unknowns are ƒ(x_(p)) and ƒ(x_(t)) for p=l or t=l. The symbolδ_(pl) is the kronecker delta. Solving these equations yields thepredicted values of ƒ(x) on the grid points in the gap.

Case II. Extrapolation

A more challenging situation occurs when ƒ(x_(l)), l>J are all unknown.In this case, for points x_(p) beyond x_(K), we specify a functionalform for the unknown grid values, including some embedded variationalparameters. It is simplest to choose linear variational parameters,e.g., $\begin{matrix}{{f(x)} \approx {\sum\limits_{\mu = 1}^{L}\quad{\alpha_{\mu}{\phi_{\mu}(x)}}}} & (12)\end{matrix}$but this is not essential, and nonlinear parameters can also be embeddedin the φ_(μ)(x). The choice of functions, φ_(μ)(x), can be guided by anyintuition or information about the physical behavior of the signal, buteven this is not necessary. This introduces additional variations of thecost function with respect to the additional parameters, so we impose${\frac{\partial C}{\partial\alpha_{\beta}} = 0},{1 \leq \beta \leq L},$and therefore obtain additional equations. We must also specify thechoice of the W_(p) when one introduces both a gap and a “tailfunction”. There is enormous freedom in how this is done, and e.g., onecan choose which points are included and which are not. In the presentstudy, we shall take W_(p)=1 for 1≦p≦K (i.e., all known data points,plus all gap-points), and W_(p)≡1 for all other points (includingtail-function points). Again, we emphasize that other choices arepossible and are under study.

For case I, our procedure leads to Equation (11) and for case II, to theequations, $\begin{matrix}{{ {\sum\limits_{p = {1 - w}}^{\min{({{l + w},K})}}{\quad 2W_{p}( {{f( x_{p} )} - {\sum\limits_{t = {p - w}}^{p + w}\quad{\delta_{M}( {x_{p} - x_{l}} }}} \middle| { {\sigma\quad} ){f( x_{l} )}} ) \times ( {\delta_{pl} - {{{\delta_{M}( {x_{p} - x_{l}} }}\sigma}} )}} ) = 0},\quad{J \leq l \leq K},} & (13) \\{{{\sum\limits_{p = {K + 1 - w}}^{K}{\quad{- 2}W_{p}( {{f( x_{p} )} - {\sum\limits_{t = {p - w}}^{p + w}\quad{\delta_{M}( {x_{p} - x_{l}} }}} \middle| { {\sigma\quad} ){f( x_{l} )}} ) \times {\sum\limits_{l = {K + 1}}^{p + w}\quad{{\delta_{M}( {x_{p} - x_{l}} )}{\phi_{\beta}( x_{l} )}}}}} = 0},\quad{1 \leq \beta \leq L}} & (14)\end{matrix}$These linear algebraic equations can be solved by any of a variety ofstandard algorithms [18]. Note that it is the well-tempered property ofthe DAFs that underlies the above algorithms. For standard interpolationalgorithms, the value on each grid point is exact and does not depend onthe values at other grid points, which means that the cost function isalways zero irrespective of functional values.

The suitability of using Hermite DAFs to pad two isolated data sets hasbeen tested for fitting one dimensional potential energy surfaces [8].To explore further the algorithm in the case where only one data set isknown, we show in FIG. 2 the extrapolation results for the arbitrarilychosen DAF-class function, $\begin{matrix}{{f(x)} = {4 + {2{\sum\limits_{j = 1}^{4}\quad{\cos({jx})}}} + {5{\exp( {- ( {x - 1} )^{2}} )}}}} & (15)\end{matrix}$Using values of the function on a discrete grid with uniform spacing,Δ≈0.024, on the domain shown in FIG. 2 (solid line), we attempt todetermine the function at 100 uniformly distributed grid points in therange [−1.2, 1.2]. The tail function used is ƒ(x)≡1, multiplied by alinear variational parameter. From FIG. 2, it is seen that the predictedresults are in almost total agreement with the actual function, for allthree DAF parameters employed, for the points between −1.2≦x≦0.2. Largererrors occur for those x values which are further away from the knowndata boundary. The source of error simply is that one is forcing thefunction to join smoothly with a constant tail function, even though theconstant is variationally determined. Had one employed the correct formfor the tail function, with a multiplicative variational factor, theresult would be visually indistinguishable for all three DAF parametersand the tail-variational constant would turn out to be essentiallyunity. It must be noted that, although we have discussed the algorithmin the context of one dimension, extending it to two or more dimensionsis straightforward. One way to do this is with a direct product form, asgiven in Equation (4). However, such a direct two dimensionalcalculation is a time and memory consuming procedure because of thelarge number of simultaneous algebraic equations that must be solved.One alternative is to consider the two dimensional patterns as a gridconstructed of many one dimensional grids, and then extrapolate eachline separately. We expect this procedure may be less accurate than thedirect two dimensional extrapolation algorithm because it only considersthe influence from one direction and neglects cross correlation.However, for many problems it produces satisfactory results and it is avery economical procedure. Additionally, cross correlation can beintroduced by DAF-fitting the complete (known plus predicted) data setusing the appropriate 2D DAF.

The well-tempered property makes the DAFs powerful computational toolsfor extrapolation of noisy data, since DAFs are low-pass filters andtherefore remove high frequency noise. In the next section, we willexplore the use of the algorithm presented here for periodicallyextending a finite, discrete segment of data which may contain noise.

PERIODIC EXTENSION

As described in the introduction to this disclosure, sometimes it isnecessary to know the boundary conditions for a data set in order toapply noncausal, zero-phase FIR filters without inducing significantaliasing. Certain other numerical analyses require that the signalcontain a number of samples equal to an integer power of 2. However, itis most often the case that the boundary conditions for the experimentaldata are unknown and the length of the data stream is fixedexperimentally and typically not subject to adjustment.

A pseudo-signal is introduced outside the domain of the nonperiodicexperimental signal in order to force the signal to satisfy periodicboundary conditions, and/or to have the appropriate number of samples.The required algorithm is similar to that for filling a gap, asdiscussed above. One can treat the period-length as a discretevariational parameter but we don't pursue this here. For a given set ofexperimental data {ƒ₁, ƒ₂, . . . , ƒ_(J−1))}, we shall force it to beperiodic, with period K, so that K−J+1 values {ƒ_(J), ƒ_(J+1), . . . ,ƒ_(K)} must be determined. Since the extended signal is periodic, thevalues {ƒ_(K+1), ƒ_(K+2), . . . , ƒ_(K+J−1)} are also, by fiat, known tobe equal to {ƒ₁, ƒ₂, . . . , ƒ_(J−1)}. Once the gap is filled in, theresulting signal can, of course, be infinitely extended as may berequired for various numerical applications.

The pseudo-signal is only used to extend the data periodically retainingessentially the same frequency distributions. The utility of the presentperiodic extension algorithm is that it provides an artificial boundarycondition for the signal without significant aliasing. The resultingsignal can be used with any filter requiring information concerning thefuture and past behavior of the signal. In this disclosure, we alsoemploy a Hermite DAF to filter out the higher frequency noise of theperiodic, extended noisy signal. For doing this, we assume that the truesignal is bandwidth limited and that the noise is mostly concentrated inthe high frequency region.

As shown in the test example extrapolation in section III, there areinfinitely many ways to smoothly connect two isolated signals using DAFswith different choices of the parameters. We require a procedure todetermine the optimum DAF parameters in a “blind” manner. Fourieranalysis is one way to proceed, but due to the structure of the HermiteDAF, we prefer to optimize the parameters while working in physicalspace rather than Fourier space. To accomplish this, we introduce ageneralized signature for both noisy data extension and filtering, whichwe define to be $\begin{matrix}{{S_{M}( {\sigma/\Delta} )} = \sqrt{\frac{\sum\limits_{n}( {{f_{DAF}( x_{n} )} - \overset{\_}{f_{DAF}}} )^{2}}{N}}} & (16)\end{matrix}$where, M and σ are Hermite DAF parameters, and {overscore (ƒDAF)} is thearithmetic average of the ƒ_(DAF)(x_(n)) (the σ/Δ→∞ of ƒ_(DAF)(x_(n))).The signature essentially measures the smoothness of the DAF filteredresult.

A typical plot of S_(M)(σ/Δ) is shown in FIG. 4(b). We first note thatit is monotonically decreasing. This is to be expected since increasingσ/Δ results in a smoother, more highly averaged signal. The second majorfeature of interest is the occurrence of a broad plateau. In this regionmost of the noise has been removed from the DAF approximation; however,the dominant portion of the true signal is still concentrated under theDAF frequency window. As a consequence the DAF approximation to thefunction is very stable in this region. As σ/Δ increases beyond theplateau the width of the DAF window in frequency no longer captures thetrue signal and as a result, the true DAF signal begins to be severelyaveraged. In the extreme, only the zero frequency remains andƒ_(DAF)(x_(n))={overscore (ƒDAF)} and hence S_(M)(σ/Δ)→0. As we discussbelow, one can usefully correlate the transition behavior with the bestDAF-approximation. The first extremely rapid decrease is due to the factthe DAF is interpolating and not well tempered. It is the region beyondthe initial rapid decrease that is important (i.e., σ/Δ≧1.5). Tounderstand the behavior in this region, we write S_(M)(σ/Δ) in the form$\begin{matrix}{{S_{M}( {\sigma/\Delta} )} = \sqrt{\frac{\begin{matrix}{{\sum\limits_{n}{( \quad ( {\Delta\quad{f( x_{n} )}} )_{DAF}^{2}}} +} \\ { {{{{2( {\Delta\quad{f( x_{n} )}} )_{DAF}( {\quad\quad} }\quad}{f_{DAF}^{(p)}( x_{n} )}} - {\overset{\_}{f_{DAF}}\quad}} ) + {( \quad {f_{DAF}^{(p)}( x_{n} )}} - {\overset{\_}{f_{DAF}}\quad}} )^{2}\end{matrix}}{N}}} & (17)\end{matrix}$where ƒ_(DAF) ^((p))(x_(n)) is the DAF approximation using a σ/Δ in themiddle of the plateau and (Δƒ(x_(n)))_(DAF)=ƒ_(DAF)(x_(n))−ƒ_(DAF)^((p))(x_(n)). The cross term averages to zero because the DAFapproximation is interpolating and hence the (Δƒ(x_(n)))_(DAF) fluctuaterapidly reflecting the presence of noise. Thus $\begin{matrix}{{S_{M}( {\sigma/\Delta} )} = \sqrt{\frac{ {{\sum\limits_{n}( {\Delta\quad{f( x_{n} )}} )_{DAF}^{2}} + {( \quad {f_{DAF}^{(p)}( x_{n} )}} - {\overset{\_}{f_{DAF}}\quad}} )^{2}}{N}}} & (18)\end{matrix}$which decreases rapidly since Σ_(n) (Δƒ(x_(n)))² _(DAF) is positive andrapidly decreasing as the high frequency noise is eliminated from thesignal. The transition into the plateau reflects a change from aninterpolative to a well tempered behavior. Although the algorithmpresented in this section only refers to periodic extensions, we stressthat this is only one possibility out of many.

NUMERICAL EXAMPLES

Two numerical examples are presented in this section to show theusefulness of our algorithm.

Case I

The first one is the extrapolation of ƒ(x)=sin(5πx/128), to which noisehas been added. The Hermite DAF parameters are M=6 for padding/extensionand M=1 for smoothing in our numerical examples. The weight W_(p), wastaken as discussed above. The values at 220 evenly spaced grid pointsare input over the range [0,219], with 50% random noise added (ƒ=ƒ×[1+random(−0.5,0.5)]. The continuation of the solid curve from points x₂₂₀to x₂₅₆ shows the function without noise. We shall predict the remaining36 points (excluding x₂₅₆ because the function there must equal thefunction at x₀) by the periodic extension algorithm presented in thisdisclosure. Because the original continuous function without noise istruly periodic, with period 256, this extension corresponds to fillingthe gap using noisy input data. The L_(∞) error and the signature forperiodic extension are plotted with respect to σ/Δ in FIGS. 4(a) and4(b) respectively. From FIG. 4(b), we see that at σ/Δ≈10.5, thetransition from the plateau to smoothing of the true signal occurs. Asis evident from FIG. 4(a), the minimum extension error also occurs atσ/Δ around 10.5. In FIG. 3, we see the comparison of the true function(solid line) from the 220th to the 256th grid point, along with theperiodic extension result (“+′” symbols). It is clearly seen that theyagree very well.

We next use the padded, extended signal for low pass filtering. We plotin FIGS. 5(a) and 5(b) the L_(∞) error and the signature of the filteredresult for one complete period as a function of σ/Δ using M=12 ratherthan M=6 in the DAF. This is done for convenience for reasons notgermane to the subject. The result is that the σ/Δ range for which theDAF is well-tempered changes and the transition from denoising to signalmodifying smoothing occurs at σ/Δ=17 (FIG. 5(b)). However, we see fromFIG. 5(a) that the L_(∞) minimum error also occurs at about the sameσ/Δ, showing the robustness of the approach with respect to the choiceof DAF parameters. In FIG. 5(c), we show the resulting smoothed,denoised sine function compared to the original true signal. Theseresults illustrate the use of the DAF procedure in accurately extractinga band-limited function using noisy input data. Because of therelatively broad nature of the L_(∞) error near the minimum, one doesnot need a highly precise value of σ/Δ.

Case II

We now consider a more challenging situation. It often happens inexperiments that the boundary condition of the experimental signal isnot periodic, and is unknown, in general. However, the signal isapproximately band-limited (i.e., in the DAF class).

To test the algorithm for this case, we use the function given inEquation (15) as an example. FIG. 6(a) shows the function, with 20%random noise in the range (−7,10) (solid line). These noisy values areassumed known at only 220 grid points in this range. Also plotted inFIG. 6(a) are the true values of the function (dashed line) on the 36points to be predicted. In our calculations, these are treated, ofcourse, as unknown and are shown here only for reference. It is clearlyseen that the original function is not periodic on the range of 256 gridpoints. We force the noisy function to be periodic by padding the valuesof the function on these last 36 points, using only the known, noisy 220values to periodically surround the gap.

As mentioned in section IV, for nonperiodic signals, the periodicextension is simply a scheme to provide an artificial boundary conditionin a way that does not significantly corrupt the frequency distributionof the underlying true signal in the sampled region. The periodicpadding signature is shown in FIG. 7. Its behavior is similar to that ofthe truly periodic signal shown in FIG. 4(b). The second rapid decreasebegins at about σ/Δ=5.2 and the periodic padding result for this DAFparameter is plotted in FIG. 6(b) along with the original noisy signal.Compared with the original noisy signal, it is seen that the signal inthe extended domain is now smoothed. In order to see explicitly theperiodic property of the extended signal and the degree to whichaliasing is avoided, we filter the noise out of the first 220 pointsusing an appropriate Hermite DAF. The L_(∞) error and the signature ofthe DAF-smoothed results are plotted in FIGS. 8(a) and 8(b)respectively. Again they correlate with each other very well. Both theminimum error and the starting point of the second rapid decrease occurat about σ/Δ=9.0, which further confirms our analysis of the behavior ofthe signature. In FIG. 9, we present the smoothed signal (solid line)along with true signal (dashed line), without any noise added. It isseen that in general, they agree with each other very well in theoriginal input signal domain.

Application of the pseudo-signal in the extended domain clearlyeffectively avoids the troublesome aliasing problem. The minor errorsobserved occur in part because of the fact that the random noisecontains not only high frequency components but also some lowerfrequency components. However, the Hermite DAF is used only as a lowpass filter here. Therefore, any low frequency noise components are leftuntouched in the resulting filtered signal. Another factor which mayaffect the accuracy of filtering is that we only use M=12. According toprevious analysis of the DAF theory, the higher the M value, the greaterthe accuracy [12], but at the expense of increasing the DAF bandwidth(σ/Δ increases). However, as M is increased, combined with theappropriate σ/Δ, the DAF-window is better able to simulate an idealband-pass filter (while still being infinitely smooth and withexponential decay in physical and Fourier space.) Here we have chosen toemploy M=12 because our purpose is simply to illustrate the use of ouralgorithm, and an extreme accuracy algorithm incorporating thisprinciples is directly achievable.

CONCLUSIONS AND DISCUSSIONS

This paper presents a DAF-padding procedure for periodically extending adiscrete segment of a signal (which is nonperiodic). The resultingperiodic signal can be used in many other numerical applications whichrequire periodic boundary conditions and/or a given number of signalsamples in one period. The power of the present algorithm is that itessentially avoids the introduction of aliasing into the true signal. Itis the well-tempered property of the DAFs that makes them robustcomputational tools for such applications. Application of an appropriatewell-tempered DAF to the periodically extended signal shows that theyare also excellent low pass filters. Two examples are presented todemonstrate the use of our algorithm. The first one is a truncated noisyperiodic function. In this case, the extension is equivalent to anextrapolation.

Our second example shows how one can perform a periodic extension of anonperiodic, noisy finite-length signal. Both examples demonstrate thatthe algorithm works very well under the assumption that the true signalis continuous and smooth. In order to determine the best DAF parameters,we introduce a quantity called the signature. It works very well bothfor extensions and low pass filtering. By examining the behavior of thesignature with respect to σ/Δ, we can determine the overall frequencydistribution of the original noisy signal working solely in physicalspace, rather than having to transform to Fourier space.

REFERENCES

-   [1] A. Nauts, R. E. Wyatt, Phys. Rev. Lett. 51, 2238 (1983).-   [2] D. Neuhauser, J. Chem. Phys. 93, 2611 (1990).-   [3] G. A. Parker, W. Zhu, Y. Huang, D. K. Hoffman, and D. J. Kouri,    Comput. Phys. Commun. 96, 27 (1996).-   [4] B. Jawerth, W. Sweldens, SIAM Rev. 36, 377 (1994).-   [5] G. Beylkin, J. Keiser, J. Comput. Phys. 132, 233 (1997).-   [6] G. H. Gunaratne, D. K. Hoffman, and D. J. Kouri, Phys. Rev. E    57, 5146 (1998).-   [7] D. K. Hoffman, G. H. Gunaratne, D. S. Zhang, and D. J. Kouri, in    preparation.-   [8] A. M. Frishman, D. K. Hoffman, R. J. Rakauskas, and D. J. Kouri,    Chem. Phys. Lett. 252, 62 (1996).-   [9] A. V. Oppenheim and R. W. Schafer, “Discrete-Time Signal    Processing” (Prentice-Hall, Inc., Englewood Cliffs, N.J., 1989).-   [10] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri, J.    Phys. Chem. 95,8299 (1991).-   [11] D. K. Hoffman, M. Arnold, and D. J. Kouri, J. Phys. Chem. 96,    6539 (1992).-   [12] J. Kouri, X. Ma, W. Zhu, B. M. Pettitt, and D. K. Hoffman, J.    Phys. Chem. 96, 9622 (1992).-   [13] D. K. Hoffman, T. L. Marchioro II, M. Arnold, Y. Huang, W. Zhu,    and D. J. Kouri, J. Math. Chem. 20, 117 (1996).-   [14] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman, J.    Chem. Phys. 107, 3239 (1997).-   [15] D. S. Zhang, G. W. Wei, D. J. Kouri, and D. K. Hoffman, Phys.    Rev. E. 56, 1197 (1998).-   [16] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman, Comput.    Phys. Commun. 111, 93(1998).-   [17] D. S. Zhang, G. W. Wei, D. J. Kouri, D. K. Hoffman, M.    Gorman, A. Palacios, and G. H. Gunaratne, Phys. Rev. E, Submitted.-   [18] W. H. Press, B. P. Flannery, S. A. Teukosky, and W. T.    Vetterling, “Numerical Recipes—The Art of Scientific Computing”    (Cambridge University Press, Cambridge, 1988).

GENERALIZED SYMMETRIC INTERPOLATING WAVELETS Introduction

The theory of interpolating wavelets based on a subdivision scheme hasattracted much attention recently [1, 9, 12, 13, 17, 22, 27, 29, 40, 42,45, 46, 47, 48, 49, 54, 55, 56, 65 and 66]. Because the digital samplingspace is exactly homomorphic to the multi scale spaces generated byinterpolating wavelets, the wavelet coefficients can be obtained fromlinear combinations of discrete samples rather than from traditionalinner product integrals. This parallel computational schemesignificantly decreases the computational complexity and leads to anaccurate wavelet decomposition, without any pre-conditioning orpost-conditioning processes. Mathematically, various interpolatingwavelets can be formulated in a biorthogonal setting.

Following Donoho's interpolating wavelet theory [12], Harten hasdescribed a kind of piecewise biorthogonal wavelet construction method[17]. Swelden independently develops this method as the well-known“lifting scheme” [56], which can be regarded as a special case of theNeville filters considered in [27]. The lifting scheme enables one toconstruct custom-designed biorthogonal wavelet transforms by justassuming a single low-pass filter (a smooth operation) withoutiterations. Theoretically, the interpolating wavelet theory is closelyrelated to the finite element technique in the numerical solution ofpartial differential equations, the subdivision scheme for interpolationand approximation, multi-grid generation and surface fitting techniques.

A new class of generalized symmetric interpolating wavelets (GSIW) aredescribed, which are generated from a generalized, window-modulatedinterpolating shell. Taking advantage of various interpolating shells,such as Lagrange polynomials and the Sinc function, etc., bell-shaped,smooth window modulation leads to wavelets with arbitrary smoothness inboth time and frequency. Our method leads to a powerful and easilyimplemented series of interpolating wavelet. Generally, this noveldesigning technique can be extended to generate other non-interpolatingmultiresolution analyses as well (such as the Hermite shell). Unlike thebiorthogonal solution discussed in [6], we do not attempt to solve asystem of algebraic equations explicitly. We first choose an updatingfilter, and then solve the approximation problem, which is a rth-orderaccurate reconstruction of the discretization. Typically, theapproximating functional is a piecewise polynomial. If we use the samereconstruction technique at all the points and at all levels of thedyadic sequence of uniform grids, the prediction will have aToplitz-like structure.

These ideas are closely related to the distributed approximatingfunctionals (DAFs) used successfully in computational chemistry andphysics [20, 21, 22, 65, 66, 67], for obtaining accurate, smoothanalytical fits of potential-energy surfaces in both quantum andclassical dynamics calculations, as well as for the calculation of thestate-to-state reaction probabilities for three-dimension (3-D)reactions. DAFs provide a numerical method for representing functionsknown only on a discrete grid of points. The underlying function orsignal (image, communication, system, or human response to some probe,etc.) can be a digital time sequence (i.e., finite in length and1-dimensional), a time and spatially varying digital sequence (including2-D images that can vary with time, 3-D digital signals resulting fromseismic measurements), etc. The general structure of the DAFrepresentation of the function, ƒ_(DAF)(x,t), where x can be a vector(i.e., not just a single variable), isƒ_(DAF)(x, t _(p))=Σ_(j)φ(x−x _(j))/|σ/Δ)ƒ(x _(j) ,t _(p))where φ(x−x_(j))|σ/Δ) is the “discrete DAF”, ƒ(x_(j),t_(p)) is thedigital value of the “signal” at time t_(p), and M and σ/Δ will bespecified in more detail below. They are adjustable DAF parameters, andfor non-interpolative DAF, they enable one to vary the behavior of theabove equation all the way from an interpolation limit, whereƒ_(DAF)(x _(j) ,t _(p))≡ƒ(x _(j) ,t _(p))(i.e., the DAF simply reproduces the input data on the grid to as highaccuracy as desired) to the well-tempered limit, whereƒ_(DAF)(x _(j) ,t _(p))≠ƒ(x _(j) ,t _(p))for function ƒ(x,t_(p))∈L²(R). Thus the well-tempered DAF does notexactly reproduce the input data. This price is paid so that instead, awell-tempered DAF approximation makes the same order error off the gridas it does on the grid (i.e., there are no special points). We haverecently shown that DAFs (both interpolating and non-interpolating) canbe regarded as a set of scaling functionals that can used to generateextremely robust wavelets and their associated biorthogonal complements,leading to a full multiresolution analysis [22, 46, 47, 48, 49, 54, 55,66, 67]. DAF-wavelets can therefore serve as an alternative basis forimproved performance in signal and image processing.

The DAF wavelet approach can be applied directly to treat boundeddomains. As shown below, the wavelet transform is adaptively adjustedaround the boundaries of finite-length signals by conveniently shiftingthe modulated window. Thus the biorthogonal wavelets in the interval areobtained by using a one-sided stencil near the boundaries. Lagrangeinterpolation polynomials and band-limited Sinc functionals inPaley-Wiener space are two commonly used interpolating shells for signalapproximation and smoothing, etc. Because of their importance innumerical analysis, we use these two kinds of interpolating shells tointroduce our discussion. Other modulated windows, such as the square,triangle, B-spline and Gaussian are under study with regard to thetime-frequency characteristics of generalized interpolating wavelets. Bycarefully designing the interpolating Lagrange and Sinc functionals, wecan obtain smooth interpolating scaling functions with an arbitraryorder of regularity.

INTERPOLATING WAVELETS

The basic characteristics of interpolating wavelets of order D discussedin reference [12] require that, the primary scaling function, φ,satisfies the following conditions.

-   -   (1) Interpolation: $\begin{matrix}        {{\phi(k)} = \{ \quad{{\begin{matrix}        {1,} & {k = 0} \\        {0,} & {k \neq 0}        \end{matrix}\quad k} \in Z} } & (19)        \end{matrix}$        where Z denotes the set of all integers.    -   (2) Self-Induced Two-Scale Relation: φ can be represented as a        linear combination of dilates and translates of itself, with a        weight given by the value of φ at k/2. $\begin{matrix}        {{\phi(x)} = {\sum\limits_{k}\quad{{\phi( {k\text{/}2} )}{\phi( {{2x} - k} )}}}} & (20)        \end{matrix}$

This relation is only approximately satisfied for some interpolatingwavelets discussed in the later sections. However, the approximation canbe made arbitrarily accurate.

-   -   (3) Polynomial Span: For an integer D≧0, the collection of        formal sums symbol ΣC_(k)φ(x−k) contains all polynomials of        degree D.    -   (4) Regularity: For a real V>0, φ is Hölder continuous of order        V.    -   (5) Localization: φ and all its derivatives through order └V┘        decay rapidly.         |φ^((r))(x)|≦A _(s)(1+|x|)^(−s) , x∈R, s>0, 0≦r≦└V┘  (21)        where └V┘ represents the maximum integer that does not exceed V.

In contrast to most commonly used wavelet transforms, the interpolatingwavelet transform possesses the following characteristics:

1. The wavelet transform coefficients are generated by the linearcombination of signal samplings,S _(j,k)=2^(−j/2)ƒ(2^(−j) k), W _(j,k)=2^(−j/2)[ƒ(2^(−j)(k+½))−(P_(j)ƒ)(2^(−j)(k+½))]  (22)instead of the convolution of the commonly used discrete wavelettransform, such asW _(j,k)=∫_(R) ψ_(j,k)(x)ƒ(2^(−j) k)dx  (23)where the scaling function, φ_(j,k)(x)=2^(j/2)φ(2^(j)x−k), and waveletfunction, ψ_(j,k)(x)=2^(j/2) ψ(2^(j)x−k), P_(j)ƒ as the interpolant2^(−j/2)Σƒ(2^(−j)k)φ_(j,k)(x).

2. A parallel-computing mode can be easily implemented. The calculationand compression of coefficients does not depend on the results of othercoefficients. For the halfband filter with length N, the calculation ofeach of the wavelet coefficients, W_(j,k), does not exceed N+2multiply/adds.

3. For a D-th order differentiable function, the wavelet coefficientsdecay rapidly.

4. In a mini-max sense, threshold masking and quantization are nearlyoptimal approximations for a wide variety of regularity algorithms.

Theoretically, interpolating wavelets are closely related to thefollowing functions:

Band-limited Shannon wavelets

The π band-limited function, φ(x)=sin(πx)/(πx)∈C^(∞) in Paley-Wienerspace, generates the interpolating functions. Every band-limitedfunction ƒ∈L²(R) can be reconstructed using the equation $\begin{matrix}{{f(x)} = {\sum\limits_{k}\quad{{f(k)}\quad\frac{\sin\quad{\pi( {x - k} )}}{\pi( {x - k} )}}}} & (24)\end{matrix}$where the related wavelet function—Sinclet is defined as (see FIG. 10)$\begin{matrix}{{\psi(x)} = \frac{{\sin\quad{\pi( {{2x} - 1} )}} - {\sin\quad{\pi( {x - {1/2}} )}}}{\pi( {x - {1/2}} )}} & (25)\end{matrix}$Interpolating fundamental splines

The fundamental polynomial spline of degree D, η^(D)(x), where D is anodd integer, has been shown by Schoenberg (1972), to be an interpolatingwavelet (see FIG. 11). It is smooth with order R=D−1, and itsderivatives through order D−1 decay exponentially [59]. Thus,η^(D)(x)=Σ_(k)α^(D)(k)β^(D)(x−k)  (26)where β^(D)(x) is the B-spline of order D defined as $\begin{matrix}{{\beta^{D}(x)} = {\sum\limits_{j = 0}^{D + 1}\quad{\frac{( {- 1} )^{j}}{D!}( \quad\begin{matrix}{D + 1} \\j\end{matrix}\quad )( {x + \frac{D + 1}{2} - j} )^{D}{U( {x + \frac{D + 1}{2} - j} )}}}} & (27)\end{matrix}$Here U is the step function $\begin{matrix}{{U(x)} = \{ \quad\begin{matrix}{0,} & {x < 0} \\{1,} & {x \geq 0}\end{matrix}\quad } & (28)\end{matrix}$and {α^(D)(k)} is a sequence that satisfies the infinite summationconditionΣ_(k)α^(D)(k)β^(D)(n−k)=δ(n)  (29)Deslauriers-Dubuc functional

Let D be an odd integer, D>0. There exist functions F_(D) such that ifF_(D) has already been defined at all binary rationals with denominator2^(j), it can be extended by polynomial interpolation, to all binaryrationals with denominator 2^(j+1), i.e. all points halfway betweenpreviously defined points [9, 13]. Specifically, to define the functionat (k+½)/2^(j) when it is already defined at all {k2^(−j)}, fit apolynomial π_(j,k) to the data (k′/2^(j), F_(D)(k′/2^(j)) fork′∈{2^(−j)[k−(D−1)/2], . . . , 2^(−j)[k+(D+1)/2]}. This polynomial isunique $\begin{matrix}{{F_{D}( \frac{k + {1/2}}{2^{j}} )} \equiv {\pi_{j,k}( \frac{k + {1/2}}{2^{j}} )}} & (30)\end{matrix}$

This subdivision scheme defines a function which is uniformly continuousat the rationals and has a unique continuous extension; F_(D) is acompactly supported interval polynomial and is regular; It is theauto-correlation function of the Daubechies wavelet of order D+1. It isat least as smooth as the corresponding Daubechies wavelets (roughlytwice as smooth).

Auto-correlation shell of orthonormal wavelets

If {hacek over (φ)} is an orthonormal scaling function, itsauto-correlation φ={hacek over (φ)}*{hacek over (φ)} (−·) is aninterpolating wavelet (FIG. 12) [40]. Its smoothness, localization andthe two-scale relation are inherited from {hacek over (φ)}. Theauto-correlations of Haar, Lamarie-Battle, Meyer, and Daubechieswavelets lead to, respectively, the interpolating Schauder,interpolating spline, C^(∞) interpolating, and Deslauriers-Dubucwavelets.

Lagrange half-band filters

Ansari, Guillemot, and Kaiser [1] used Lagrange symmetric halfband FIRfilters to design the orthonormal wavelets that express the relationbetween the Lagrange interpolators and Daubechies wavelets [7]. Theirfilter corresponds to the Deslauriers-Dubuc wavelet of order D=7 (2M−1),M=4. The transfer function of the halfband symmetric filter h is givenbyH(z)=½+zT(z ²)  (31)where T is the trigonometric polynomial. Except for h(0)=½, at everyeven integer lattice point h(2n)=0, n≠0, n∈Z. The transfer function ofthe symmetric FIR filter h(n)=h(−n), has the form $\begin{matrix}{{H(z)} = {{1/2} + {\sum\limits_{n = 1}^{M}\quad{{h( {{2n} - 1} )}( {z^{1 - {2n}} + z^{{2n} - 1}} )}}}} & (32)\end{matrix}$

The concept of an interpolating wavelet decomposition is similar to“algorithm a trous”, the connection having been found by Shensa [42].The self-induced scaling and interpolation conditions are the mostimportant characteristics of interpolating wavelets. From the followingequationƒ(x)=Σ_(n)ƒ(n)φ(x−n)  (33)and Equation (19), the approximation to the signal is exact on thediscrete sampling points, which does not hold in general for commonlyused non-interpolating wavelets.

GENERALIZED INTERPOLATING WAVELETS

Interpolating wavelets with either a Lagrange polynomial shell or Sincfunctional shell are discussed in detail. We call these kinds of windowmodulated wavelets generalized interpolating wavelets, because they aremore convenient to construct, processing and extend to higherdimensional spaces.

Generalized Lagrange Wavelets

Three kinds of interpolating Lagrange wavelets, Halfband Lagrangewavelets, B-spline Lagrange wavelets and Gaussian-Lagrange DAF wavelets,are studied here as examples of the generalized interpolating wavelets.

Halfband Lagrange wavelets can be regarded as extensions of the Dubucinterpolating functionals [9, 13], the auto-correlation shell waveletanalysis [40], and halfband filters [1]. B-spline Lagrange Wavelets aregenerated by a B-spline-windowed Lagrange functional which increases thesmoothness and localization properties of the simple Lagrange scalingfunction and its related wavelets. Lagrange Distributed ApproximatingFunctionals (LDAF)-Gaussian modulated Lagrange polynomials, have beensuccessfully applied for numerically solving various linear andnonlinear partial differential equations. Typical examples includeDAF-simulations of 3-dimensional reactive quantum scattering and thesolution of a 2-dimensional Navier-Stokes equation with non-periodicboundary conditions. In terms of a wavelet analysis, DAFs can beregarded as particular scaling functions (wavelet-DAFs) and theassociated DAF-wavelets can be generated in a number of ways [20, 21,22, 65, 66, 67].

Halfband Lagrange Wavelets

A special case of halfband filters can be obtained by choosing thefilter coefficients according to the Lagrange interpolation formula. Thefilter coefficients are given by $\begin{matrix}{{h( {{2n} - 1} )} = \frac{( {- 1} )^{n + M - 1}{\prod\limits_{m = 1}^{2M}\quad( {M + {1/2} - m} )}}{{( {M - n} )!}{( {M + n - 1} )!}( {{2n} - 1} )}} & (34)\end{matrix}$These filters have the property of maximal flatness in Fourier space,possessing a balance between the degree of flatness at zero frequencyand the flatness at the Nyquist frequency (half sampling).

These half-band filters can be utilized to generate the interpolatingwavelet decomposition, which can be regarded as a class of theauto-correlated shell of orthogonal wavelets, such as the Daubechieswavelets [7]. The interpolating wavelet transform can also be extendedto higher order cases using different Lagrange polynomials, as [40]$\begin{matrix}{{P_{{2n} - 1}(x)} = {\prod\limits_{{m = {{- M} + 1}},{m \neq n}}^{M}\quad\frac{x - ( {{2m} - 1} )}{( {{2n} - 1} ) - ( {{2m} - 1} )}}} & (35)\end{matrix}$The predictive interpolation can be expressed as $\begin{matrix}{{{\Gamma\quad{S_{j}(i)}} = {\sum\limits_{n = 1}^{M}\quad{{P_{{2n} - 1}(0)}\lbrack {{S_{j}( {i + {2n} - 1} )} + {S_{j}( {i - {2n} + 1} )}} \rbrack}}},{i = {{2k} + 1}}} & (36)\end{matrix}$where Γ is a projection and S_(j) is the jth layer low-passcoefficients. This projection relation is equivalent to the subbandfilter response ofh(2n−1)=P _(2n−1)(0)  (37)The above-mentioned interpolating wavelets can be regarded as theextension of the fundamental Deslauriers-Dubuc interactive sub-divisionscheme, which results when M=2. The order of the Lagrange polynomial isD=2M−1=3 (FIG. 15(a)).

It is easy to show that an increase of the Lagrange polynomial order D,will introduce higher regularity for the interpolating functionals (FIG.16(a)). When D→+∞, the interpolating functional tends to a band-limitedSinc function and its domain of definition is on the real line. Thesubband filters generated by Lagrange interpolating functionals satisfythe properties:

(1) Interpolation: h(ω)+h(ω+π)=1

(2) Symmetry: h(ω)=h(−ω)

(3) Vanishing Moments: ∫_(R)x^(p)φ(x)dx=δ_(p)

-   -   Donoho outlines a basic subband extension to obtain a perfect        reconstruction. He defines the wavelet function as        ψ(x)=φ(2x−1)  (38)        The biorthogonal subband filters can be expressed as         {tilde over (h)}(ω)=1, g(ω)=e ^(−iω) , {tilde over (g)}(ω)=e        ^(−iω) {overscore (h(ω+π))}  (39)        However, the Donoho interpolating wavelets have some drawbacks.        Because the low-pass coefficients are generated by a sampling        operation only, as the decomposition layer increases, the        correlation between low-pass coefficients become weaker. The        interpolating (prediction) error (high-pass coefficients)        strongly increases, which is deleterious to the efficient        representation of the signal. Further, it can not be used to        generate a Riesz basis for L²(R) space.

Swelden has provided an efficient and robust scheme [56] forconstructing biorthogonal wavelet filters. His approach can be utilizedto generate high-order interpolating Lagrange wavelets with higherregularity. As FIG. 13 shows, P₀ is the interpolating predictionprocess, and the P₁ filter is called the updating filter, used to smooththe down-sampling low-pass coefficients. If we choose P₀ to be the sameas P₁, then the new interpolating subband filters can be depicted as$\begin{matrix}\{ \begin{matrix}{{h_{1}(\omega)} = {h(\omega)}} \\{{{\overset{\sim}{h}}_{1}(\omega)} = {1 + {{\overset{\sim}{g}(\omega)}\overset{\_}{P( {2\quad\omega} )}}}} \\{{g_{1}(\omega)} = {{\mathbb{e}}^{{- i}\quad\omega} - {{h(\omega)}{P( {2\quad\omega} )}}}} \\{{{\overset{\sim}{g}}_{1}(\omega)} = {\overset{\sim}{g}(\omega)}}\end{matrix}\quad  & (40)\end{matrix}$The newly developed filters h₁, g₁, {tilde over (h)}, and {tilde over(g)} also generate the biorthogonal dual pair for a perfectreconstruction. Examples of biorthogonal lifting wavelets withregularity D=3 are shown in FIG. 14. FIG. 15 gives the correspondingFourier responses of the equivalent subband decomposition filters.B-Spline Lagrange Wavelets

Lagrange polynomials are natural interpolating expressions forfunctional approximations. Utilizing a different expression for theLagrange polynomials, we can construct other forms of usefulinterpolating wavelets as follows. We define a class of symmetricLagrange interpolating functional shells as $\begin{matrix}{{P_{M}(x)} = {\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}} & (41)\end{matrix}$It is easy to verify that this Lagrange shell also satisfies theinterpolating condition on discrete, integer points, $\begin{matrix}{{P_{M}(k)} = \{ \begin{matrix}{1,} & {k = 0} \\{0,} & {otherwise}\end{matrix} } & (42)\end{matrix}$However, simply defining the filter response ash(k)=P(k/2)/2, k=−M, M  (43)leads to non-stable interpolating wavelets, as shown in FIG. 16.

Including a smooth window, which vanishes at the zeros of the Lagrangepolynomial, will lead to more regular interpolating wavelets andequivalent subband filters (as shown in FIGS. 16 and 17). We select awell-defined B-spline function as the weight window. Then the scalingfunction (mother wavelet) can be defined as an interpolating B-SplineLagrange functional (BSLF) $\quad\begin{matrix}{{\phi_{M}(x)} = {{\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}{P_{M}(x)}} = {\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}{\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}}}} & (44)\end{matrix}$where N is the B-spline order and, η is the scaling factor to controlthe window width. To ensure coincidence of the zeroes of the B-splineand the Lagrange polynomial, we set2M=η×(N+1)  (45)To ensure the interpolation condition, the B-spline envelope degree Mmust be odd number. It is easy to show that if B-spline order is N=4k+1,η can be any odd integer (2k+1); if N is an even integer, then η canonly be 2. When N=4k−1, we can not construct an interpolating shellusing the definition above. From the interpolation and self-inducedscaling properties of the interpolating wavelets, it is easy to verifythath(k)=φ_(M)(k/2)/2, k=−2M+1, 2M−1   (46)Gaussian-Lagrange DAF Wavelets

We can also select a class of distributed approximationfunctional—Gaussian-Lagrange DAFs (GLDAF) as our basic scaling functionto construct interpolating wavelets as: $\quad\begin{matrix}{{\phi_{M}(x)} = {{{W_{\sigma}(x)}{P_{M}(x)}} = {{W_{\sigma}(x)}{\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}}}} & (47)\end{matrix}$where W_(σ)(x) is a window function. It is chosen to be a Gaussian,W _(σ)(x)=e ^(−x) ² ^(/2σ) ²   (48)because it satisfies the minimum frame bound condition in quantumphysics. Here σ is a window width parameter, and P_(M)(x) is theLagrange interpolation kernel. The DAF scaling function has beensuccessfully introduced as an efficient and powerful grid method forquantum dynamical propagations [40]. Using Swelden's lifting scheme[32], a wavelet basis is generated. The Gaussian window in ourDAF-wavelets efficiently smoothes out the Gibbs oscillations, whichplague most conventional wavelet bases. The following equation shows theconnection between the B-spline window function and the Gaussian window[34]: $\begin{matrix}{{\beta^{N}(x)} \cong {\sqrt{\frac{6}{\pi( {N + 1} )}}{\exp( \frac{{- 6}\quad x^{2}}{N + 1} )}}} & (49)\end{matrix}$for large N. As in FIG. 21, if we choose the window widthσ=η√{square root over ((N+1)/12)}  (50)the Gaussian-Lagrange wavelets generated by the lifting scheme will besimilar to the B-spline Lagrange wavelets. Usually, theGaussian-Lagrange DAF displays a slightly better smoothness and morerapid decay than the B-spline Lagrange wavelets. If we select moresophisticated window shapes, such as those popular in engineering(Bartlett, Hanning, Hamming, Blackman, Chebychev, and Bessel windows),the Lagrange wavelets can be generalized further. We shall call theseextensions Bell-windowed Lagrange wavelets.Generalized Sinc Wavelets

As we have mentioned above, the π band-limited Sinc function,φ(x)=sin(πx)/(πx)C ^(∞)  (51)in Paley-Wiener space, constructs an interpolating function. Every πband-limited function ƒ∈L²(R) can be reconstructed by the equation$\begin{matrix}{{f(x)} = {\sum\limits_{k}{{f(k)}\quad\frac{\sin\quad{\pi( {x - k} )}}{\pi( {x - k} )}}}} & (52)\end{matrix}$where the related wavelet function—Sinclet is defined as (see FIG. 10)$\begin{matrix}{{\psi(x)} = \frac{{\sin\quad{\pi( {{2x} - 1} )}} - {\sin\quad{\pi( {x - {1/2}} )}}}{\pi( {x - {1/2}} )}} & (53)\end{matrix}$The scaling Sinc function is the well-known ideal low-pass filter, whichpossesses the ideal square filter response as $\begin{matrix}{{H(\omega)} = \{ \begin{matrix}{1,} & {{\omega } \leq {\pi/2}} \\{0,} & {{\pi/2} < {\omega } \leq \pi}\end{matrix} } & (54)\end{matrix}$Its impulse response can be generated ash[k]=∫ _((−π/2,π/2)) e ^(jkω) dω/2π=sin(πk/2)/(πk)  (55)The so-called half-band filter possess a non-zero impulse only at theodd integer sampler, h(2k+1), while at even integers, h[2k]=0 unless ak=0.

However, this ideal low-pass filter is never used in application. Sincethe digital filter is an IIR (infinite impulse response) solution, itsuse as a digital cutoff FIR (finite impulse response) will produce Gibbsphenomenon (overshot effect) in Fourier space, which degrades thefrequency resolution (FIG. 20). The resulting compactly supported Sincscaling and wavelet functions, as well as their biorthogonal dualscaling and wavelet functions, are shown in FIG. 21. We see that theregularity of the cutoff Sinc is obviously degraded with a fractal-likeshape, which leads to poor time localization.

B-Spline Sinc Wavelets

Because the ideal low-pass Sinc wavelet can not be implemented “ideally”by FIR (finite impulse response) filters, to eliminate the cutoffsingularity, a windowed weighting technique is employed to adjust thetime-frequency localization of the Sinc wavelet analysis. To begin, wedefine a symmetric Sinc interpolating functional shell as$\begin{matrix}{{P(x)} = \frac{\sin( {\pi\quad{x/2}} )}{\pi\quad x}} & (56)\end{matrix}$Utilizing a smooth window, which vanishes gradually at the exact zerosof the Sinc functional, will lead to more regular interpolating waveletsand equivalent subband filters (as shown in FIGS. 22 and 23). Forexample, we illustrate using a well-defined B-spline function as theweight window. Then the scaling function (mother wavelet) can be definedas an interpolating B-spline Sinc functional (BSF) $\quad\begin{matrix}{{\phi_{M}(x)} = {{\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}{P(x)}} = {\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}\frac{\sin( {\pi\quad{x/2}} )}{\pi\quad x}}}} & (57)\end{matrix}$where N is the B-spline order and, η is the scaling factor to controlthe window width. To ensure the coincidence of the zeroes of theB-spline and the Sinc shell, we set2M+1=η×(N+1)/2   (58)To maintain the interpolation condition, h(2k)=0, k≠0, it is easy toshow that when the B-spline order N=4k+1, η may be any odd integer(2k+1). If N is an even integer, then η can only be 2. FIG. 24 shows amother wavelet comparison for a B-spline Sinc and a Guassian Sinc forN=4 and η2. When N=4k−1, we can not construct interpolating shell usingthe definition above. The admissibility condition can be expressed as$\begin{matrix}\{ \begin{matrix}{{\eta = 2},} & {N = {2\quad i}} \\{{\eta = {{2\quad k} + 1}},} & {N = {{4\quad i} + 1}}\end{matrix}  & (59)\end{matrix}$From the interpolation relation $\begin{matrix}{{\phi(k)} = \{ {\begin{matrix}{1,} & {k = 0} \\{0,} & {k \neq 0}\end{matrix},{k \in Z}} } & (60)\end{matrix}$and the self-induced two-scale relation $\begin{matrix}{{\phi(x)} = {\sum\limits_{k}{{\phi( {k/2} )}{\phi( {{2\quad x} - k} )}}}} & (61)\end{matrix}$it is easy to show thath(k)=φ_(M)(k/2)/2, k=−2M+1, 2M−1   (62)Gaussian-Sinc DAF Wavelets

We can also select a class of distributed approximation functionals,i.e., the Gaussian-Sinc DAF (GSDAF) as our basic scaling function toconstruct interpolating scalings, $\quad\begin{matrix}{{\phi_{M}(x)} = {{{W_{\sigma}(x)}{P(x)}} = {{W_{\sigma}(x)}\frac{\sin( {\pi\quad{x/2}} )}{\pi\quad x}}}} & (64)\end{matrix}$where W_(σ)(x) is a window function which is selected as a Gaussian,W _(σ)(x)=e ^(−x) ² ^(/2σ) ²   (64)Because it satisfies the minimum frame bound condition in quantumphysics, it significantly improves the time-frequency resolution of theWindowed-Sinc wavelet. Here σ is a window width parameter, and P(x) isthe Sinc interpolation kernel. This DAF scaling function has beensuccessfully used in an efficient and powerful grid method for quantumdynamical propagations [40]. Moreover, the Hermite DAF is known to beextremely accurate for solving the 2-D harmonic oscillator, forcalculating the eigenfunctions and eigenvalues of the Schrodingerequation. The Gaussian window in our DAF-wavelets efficiently smoothesout the Gibbs oscillations, which plague most conventional waveletbases. The following equation shows the connection between the B-splineand the Gaussian windows [34]: $\begin{matrix}{{\beta^{N}(x)} \cong {\sqrt{\frac{6}{\pi( {N + 1} )}}{\exp( \frac{{- 6}\quad x^{2}}{N + 1} )}}} & (65)\end{matrix}$for large N. As in FIG. 15, if we choose the window widthσ=η√{square root over ((N+1)/12)}  (66)the Gaussian Sinc wavelets generated by the lifting scheme will besimilar to the B-spline Sinc wavelets. Usually, the Gaussian Sinc DAFdisplays a slightly better smoothness and rapid decay than the B-splineLagrange wavelets. If we select more sophisticated window shapes, theSinc wavelets can be generalized further. We call these extensionsBell-windowed Sinc wavelets. The available choices can be any of thepopularly used DFT (discrete Fourier transform) windows, such asBartlett, Hanning, Hamming, Blackman, Chebychev, and Besel windows.

ADAPTIVE BOUNDARY ADJUSTMENT

The above mentioned generalized interpolating wavelet is defined on thedomain C(R). Many engineering applications involve finite lengthsignals, such as image and isolated speech segments. In general, we candefine these signals on C[0,1]. One could set the signal equal to zerooutside [0,1], but this introduces an artificial “jump” discontinuity atthe boundaries, which is reflected in the wavelet coefficients. It willdegrade the signal filtering and compression in multi scale space.Developing wavelets adapted to “life on an interval” is useful.Periodization and symmetric periodization are two commonly used methodsto reduce the effect of edges. However, unless the finite length signalhas a large flat area around the boundaries, these two methods cannotremove the discontinuous effects completely [4,6,11].

Dubuc utilized an iterative interpolating function, F_(D) on the finiteinterval to generate an interpolation on the set of dyadic rationalsD_(j). The interpolation in the neighborhood of the boundaries istreated using a boundary-adjusted functional, which leads to regularityof the same order as in the interval. This avoids the discontinuity thatresults from periodization or extending by zero. It is well known thatthis results in weaker edge effects, and that no extra waveletcoefficients (to deal with the boundary) have to be introduced, providedthe filters used are symmetric.

We let K_(j) represent the number of coefficients at resolution layer j,where K_(j)=2^(j). Let 2^(j)>2D+2, define the non-interactingdecomposition. If we let j₀ hold the non-interaction case 2^(j0)>2D+2,then there exist functions φ_(j,k) ^(Interval), ψ_(j,k) ^(Interval),such that for all ƒ∈C[0,1], $\begin{matrix}{f = {{\sum\limits_{k = 0}^{2^{j\quad 0} - 1}{{S_{j_{0}}(k)}\phi_{j_{0,k}}^{Interval}}} + {\sum\limits_{j \geq j_{0}}\quad{\sum\limits_{k = 0}^{2^{j} - 1}{{W_{j}(k)}\psi_{j,k}^{Interval}}}}}} & (67)\end{matrix}$The φ_(j,k) ^(Interval), ψ_(j,k) ^(Interval), are called the intervalinterpolating scalings and wavelets, which satisfy the interpolationconditions $\begin{matrix}\{ \begin{matrix}{{{\phi_{j,k}^{Interval}( {2^{- j}n} )} = {2^{j/2}\delta_{k,n}}},} & {0 \leq n < K_{j}} \\{{{\psi_{j,k}^{Interval}( {2^{{- j} - 1}n} )} = {2^{j/2}\delta_{{{2\quad k} + 1},n}}},} & {0 \leq n < K_{j + 1}}\end{matrix}  & (68)\end{matrix}$The interval scaling is defined as $\begin{matrix}{\phi_{j,k}^{Interval} = \{ \begin{matrix}{\phi_{j,k}^{Left},{0 \leq k \leq D}} \\{{\phi_{j,k}}_{\lbrack{0,1}\rbrack},{D < k < {2^{j} - D - 1}}} \\{\phi_{j,k}^{Right},{{2^{j} - D - 1} \leq k < 2^{j}}}\end{matrix} } & (69)\end{matrix}$where φ_(j,k)|_([0,1]) is called the “inner-scaling” which is identicalto the fundamental interpolating function, and φ_(j,k) ^(Right) andφ_(j,k) ^(Left) are the “left-boundary” and the “right-boundary”scalings, respectively. Both are as smooth as φ_(j,k)|_([0,1]). Intervalwavelets are defined as $\begin{matrix}{\psi_{j,k}^{Interval} = \{ \begin{matrix}{\psi_{j,k}^{Left},{0 \leq k < \lfloor {D/2} \rfloor}} \\{{\psi_{j,k}}_{\lbrack{0,1}\rbrack},{\lfloor {D/2} \rfloor \leq k < {2^{j} - \lfloor {D/2} \rfloor}}} \\{\psi_{j,k}^{Right},{{2^{j} - \lfloor {D/2} \rfloor} \leq k < 2^{j}}}\end{matrix} } & (70)\end{matrix}$ψ_(j,k)|_([0,1]) is the inner-wavelet, and ψ_(j,k) ^(Left) and ψ_(j,k)^(Right) are the left and right-boundary wavelets, respectively, whichare of the same order regularity as the inner-wavelet [13].

The corresponding factors for the Deslauriers-Dubuc interpolatingwavelets are M=2, and the order of the Lagrange polynomial is D=2M−1=3.The interpolating wavelet transform can be extended to high order casesby two kinds of Lagrange polynomials, where the inner-polynomials aredefined as [14] $\begin{matrix}{{P_{{2n} - 1}(x)} = {\prod\limits_{{m = {{- M} + 1}},{m \neq n}}^{M}\quad\frac{x - ( {{2m} - 1} )}{( {{2n} - 1} ) - ( {{2m} - 1} )}}} & (71)\end{matrix}$This kind of polynomial introduces the interpolation in the intervalsaccording to $\begin{matrix}{{P_{j}{S(i)}} = {{\sum\limits_{n = 1}^{M}\quad{{{P_{{2n} - 1}(0)}\lbrack {{S_{j}( {i + {2n} - 1} )} + {S_{j}( {i - {2n} + 1} )}} \rbrack}\quad i}} = {{2k} + 1}}} & (72)\end{matrix}$and the boundary polynomials are $\begin{matrix}{{{L_{d}(x)} = {\prod\limits_{{m = 0},{m \neq d}}^{{2M} - 1}\quad\frac{x - m}{d - m}}},{0 \leq d \leq D}} & (73)\end{matrix}$which introduce the adjusted interpolation on the two boundaries of theintervals. That is, $\begin{matrix}{{{P_{j}{S(i)}} = {\sum\limits_{d = 0}^{D}\quad{{L_{d}( {i/2} )}\quad{S_{j}( {i + {2d} - 1} )}}}},\quad{i = {{2k} + 1}},{0 \leq k \leq {\lfloor {D - 1} \rfloor/2}}} & (74)\end{matrix}$The left boundary extrapolation outside the intervals is defined as$\begin{matrix}{{{P_{j}{S( {- 1} )}} = {\sum\limits_{d = 0}^{D}\quad{{L_{d}( {{- 1}/2} )}\quad{S_{j}( {2d} )}}}},\quad{i = {{2k} + 1}},{0 \leq k \leq {\lfloor {D - 1} \rfloor/2}}} & (75)\end{matrix}$and the right boundary extrapolation is similar to the above. Theboundary adjusted interpolating scaling is shown in FIG. 25.

Although Dubuc shows the interpolation is almost twice differentiable,there still is a discontinuity in the derivative. In this disclosure, aDAF-wavelet based boundary adjusted algorithm is introduced. Thistechnique can produce an arbitrary smooth derivative approximation,because of the infinitely differentiable character of the Gaussianenvelope. The boundary-adjusted scaling functionals are generated asconveniently as possible just by window shifting and satisfy thefollowing equation.φ_(m)(x)=W(x−2m)P(x), m=−└(M−1)/2┘, └(M−1)/2┘  (76)where φ_(m)(x) represents different boundary scalings, W(x) is thegeneralized window function and P(x) is the symmetric interpolatingshell. When m>0, left boundary functionals are generated; when m<0, weobtain right boundary functionals. The casem=0 represents the innerscalings mentioned above.

One example for a Sinc-DAF wavelet is shown in FIG. 26. We choose thecompactly-supported length of the scaling function to be the same as thehalfband lagrange wavelet. It is easy to show that our newly developedboundary scaling is smoother than the commonly used Dubuc boundaryinterpolating functional. Thus it will generate a more stable boundaryadjusted representation for finite-length wavelet transforms, as well asa better derivative approximation around the boundaries. FIG. 27 is theboundary filter response comparison between the halfband Lagrangewavelet and our DAF wavelet. It is easy to establish that our boundaryresponse decreases the overshoot of the low-pass band filter, and so ismore stable for boundary frequency analysis and approximation.

APPLICATIONS OF GSIWs

Eigenvalue Solution of 2D Quantum Harmonic Oscillator

As discussed in Ref. [22], a standard eigenvalue problem of theSchrodinger equation is that of the 2D harmonic oscillator,$\begin{matrix}{{\lbrack {{{- \frac{h^{2}}{2\quad m}}{\sum\limits_{i = 1}^{2}\frac{\partial^{2}}{\partial x_{i}^{2}}}} + {\frac{1}{2}( {x_{1}^{2} + x_{2}^{2}} )}} \rbrack{\Phi_{k}( {x_{1},x_{2}} )}} = {E_{k}{\Phi_{k}( {x_{1},x_{2}} )}}} & (77)\end{matrix}$Here Φ_(k) and E_(k) are the kth eigenfunction and eigenvaluerespectively. The eigenvalues are given exactly byE _(k) ₁ _(,k) ₂ =1+k ₁ +k ₂, 0≦k<∞,0≦k ₁ ≦k ₂   (78)with a degeneracy (k_(d)=k+1) in each energy level E_(k)=1+k. The 2Dversion of the wavelet DAF representation of the Hamiltonian operatorwas constructed and the first 21 eigenvalues and eigenfunctions obtainedby subsequent numerical diagonalization of the discrete Sinc-DAFHamiltonian. As shown in Table 1, all results are accurate to at least10 significant figures for the first 16 eigenstates. It is evident thatDAF-wavelets are powerful for solving eigenvalue problems.

TABLE 1 Eigenvalues of the 2D harmonic oscillator k = k_(x) + k_(y)k_(d) Exact Solution Sinc-DAF Calculation 0 1 1 0.99999999999835 1 2 21.99999999999952 1.99999999999965 2 3 3 2.999999999998962.99999999999838 2.99999999999997 3 4 4 3.999999999999433.99999999999947 3.99999999999986 3.99999999999994 4 5 54.99999999999907 4.99999999999953 4.99999999999989 5.000000000006745.00000000000813 5 6 6 5.99999999999982 6.000000000000186.00000000000752 6.00000000000801 6.00000000011972 6.00000000012005Target Extraction

Military target extraction, including such applications ashigh-resolution radar (aerial photograph) imaging, radar echo, andremote sense detection, is a difficult subject. National DefenseAgencies (e.g. the Navy, Army, and Air Force) have great interest intechnical advances for reconnaissance, earlier warning and targetrecognition. Some of our DAF-wavelet schemes represent a significantbreakthrough for these very difficult tasks. Compared with othermethods, our algorithm possesses a very high resolution for targetlocalization and high efficiency for clutter/noise suppression, as wellas computational efficiency.

Detecting a military target in a low luminance environment ischallenging work for image processing. To improve the targetdiscrimination, the visibility of differences between a pair of imagesis important for modern image restoration and enhancement. We constructa method for detectability using a multiple channel enhancementtechnique. The images were captured on a color monitor at a viewingdistance giving 95 pixels per degree of visual angle and an image sizeof 5.33*5.05 degrees. The mean luminance of the images was about 10cd/m²[39]. Using our newly developed visual enhancement techniques, visualtargets can be extracted very accurately in a low-luminance environmentfor detection and warning. The technique combines the response of humanvision system (HVS) with multiresolution enhancement and restorationmethods. The simulation of tank-target detection in a low-luminanceenvironment is shown in FIG. 28.

Image Filtering

Image de-noising is a difficult problem for signal processing. Due tothe complicated structure of image and background noise, an optimalfiltering technique does not currently exist. Generally, the possiblenoise sources include photoelectric exchange, photo spots, imagecommunication error, etc. Such noise causes the visual perception togenerate speckles, blips, ripples, bumps, ringing and aliasing. Thenoise distortion not only affects the visual quality of images, but alsodegrades the efficiency of data compression and coding. De-noising andsmoothing are extremely important for image processing.

We use a DAF-wavelet to generate a more efficient,human-vision-system-based image processing technique, which processesthe advantages of 1) long range de-correlation for convenience ofcompression and filtering; 2) high perceptual sensitivity androbustness; 3) filtering that takes account of the human visualresponse. It therefore can enhance the most important visualinformation, such as edges, while suppressing the large scale of flatregions and background; 4) it can be carried out with real-timeprocessing.

Biorthogonal interpolating wavelets and corresponding filters areconstructed based on Gauss-Lagrange distributed approximatingfunctionals (DAFs). The utility of these DAF wavelets and filters istested for digital image de-noising in combination with a novel blindrestoration technique. This takes account of the response of humanvision system so as to remove the perceptual redundancy and obtainbetter visual performance in image processing. The test results for acolor photo are shown in FIG. 29. It is evident that our Color VisualGroup Normalization technique yields excellent contrast andedge-preservation and provides a natural color result for the restoredimage [48].

Imaging Enhancement

Mammograms are complex in appearance and signs of early disease areoften small and/or subtle. Digital mammogram image enhancement isparticularly important for solving storage and logistics problems, andfor the possible development of an automated-detection expert system.The DAF-wavelet based mammogram enhancement is implemented in thefollowing manner. First we generate a perceptual lossless quantizationmatrix Q_(j,m) to adjust the original transform coefficients C_(j,m)(k).This treatment provides a simple human-vision-based threshold techniquefor the restoration of the most important perceptual information in animage. For grayscale image contrast stretching, we appropriatelynormalize the decomposition coefficients according to the length scale,L, of the display device [16] so that they fall in the interval of [0,1]of the device frameNC _(j,m)(k)=Q _(j,m) C _(j,m)(k)/L.  (79)We then use a nonlinear mapping to obtain the desired contraststretching{overscore (NC _(j,m) )}=γ _(j,m) X _(j,m)(NC _(j,m))  (80)where the constant γ_(j,m) and function X_(j,m) are appropriately chosenso that the desired portion of the grayscale gradient is stretched orcompressed.

To test our new approach, low-contrast and low quality breast mammogramimages are employed. A typical low quality front-view image is shown inFIG. 30(a). The original image is coded at 512×512 pixel size with 2bytes/pixel and 12 bits of gray scale. We have applied our edgeenhancement normalization and device-adapted visual group normalization.As shown in FIG. 30(b), and FIG. 30(c), there is a significantimprovement in both the edge representation and image contrast. Inparticular, the domain and internal structure of high-density cancertissues are more clearly displayed. FIG. 31(a) is an original 1024×1024side-view breast image which has been digitized to a 200 micron pixeledge with 8 bits of gray scale. The enhanced image result is shown inFIG. 31(b). In this case we again obtain a significant improvement inimage quality as described herein.

CONCLUSION

In summary, a new class of wavelets—generalized symmetric interpolatingwavelets were described, which are generated by a window modulatedinterpolating shell. Due to the absence of a complicated factorizationprocess, this kind of interpolating wavelet is easily implemented andpossesses very good characteristics in both time (space) and spectraldomains. The stable boundary adjustment can be generated by a windowshifting operation only. It overcomes the overshoot of the boundaryresponse introduced by other boundary processing, such as Dubuc Lagrangewavelet and Daubechies boundary filters. Many successful applications ofDAF-wavelets have been reported to illustrate its practicality and itsmathematical behavior.

REFERENCES

-   [1] R. Ansari, C. Guillemot, and J. F. Kaiser, “Wavelet construction    using Lagrange halfband filters,” IEEE Trans. CAS, vol. 38, no. 9,    pp. 1116-1118, 1991.-   [2] R. Baraniuk, D. Jones, “Signal-dependent time-frequency analysis    using a radially Gaussian kernel,” Signal Processing, Vol. 32, pp.    263-284, 1993.-   [3] C. M. Brislawn, “Preservation of subband symmetry in multirate    signal coding,” IEEE Trans. SP, vol. 43, no. 12, pp. 3046-3050,    1995.-   [4] C. K. Chui, An Introduction to Wavelets, Academic Press, New    York, 1992.-   [5] C. K. Chui, Wavelets: A Tutorial in Wavelet Theory and    Applications, Academic Press, New York, 1992.-   [6] A. Cohen, I. Daubechies, and J. C. Feauveau, “Biorthogonal bases    of compactly supported wavelets,” Comm. Pure Appl. Math., Vol. 45,    pp. 485-560, 1992.-   [7] I. Daubechies, “Orthonormal bases of compactly supported    wavelets”, Comm. Pure and Appl. Math., vol. 41, no. 11, pp. 909˜996,    1988.-   [8] I. Daubechies, “The wavelet transform, time-frequency    localization and signal analysis,” IEEE Trans. Inform. Theory, Vol.    36, No. 5, pp. 961-1003, September 1990.-   [9] G. Deslauriers, S. Dubuc, “Symmetric iterative interpolation    processes,” Constructive Approximations, vol. 5, pp. 49-68, 1989.-   [10] A. P. Dhawan and E. Le Royer, “Mammographic feature enhancement    by computerized image processing,” Comput. Methods and Programs in    Biomed., vol. 27, no. 1, pp. 23-35, 1988.-   [11] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.    Information Theory, vol. 41, no. 3, pp. 613˜627, 1995.-   [12] D. L. Donoho, “Interpolating wavelet transform,” Preprint,    Stanford Univ., 1992.-   [13] S. Dubuc, “Interpolation through an iterative scheme”, J. Math.    Anal. and Appl., vol. 114, pp. 185˜204, 1986.-   [14] A. Frishman, D. K. Hoffman, D. J. Kouri, “Distributed    approximating functional fit of the H3 ab initio potential-energy    data of Liu and Siegbahn,” J. Chemical Physics, Vol. 107, No. 3, pp.    804-811, July 1997.-   [15] L. Gagnon, J. M. Lina, and B. Goulard, “Sharpening enhancement    of digitized mammograms with complex symmetric Daubechies wavelets,”    preprint.-   [16] R. Gordon and R. M. Rangayan, “Feature enhancement of film    mammograms using fixed and adaptive neighborhoods,” Applied Optics,    vol. 23, no. 4, pp. 560-564, February 1984.-   [17] A. Harten, “Multiresolution representation of data: a general    framework,” SIAM J. Numer. Anal., vol. 33, no. 3, pp. 1205-1256,    1996.-   [18] C. Herley, M. Vetterli, “Orthogonal time-varying filter banks    and wavelet packets,” IEEE Trans. SP, Vol. 42, No. 10, pp.    2650-2663, October 1994.-   [19] C. Herley, Z. Xiong, K. Ramchandran and M. T. Orchard, “Joint    Space-frequency Segmentation Using Balanced Wavelet Packets Trees    for Least-cost Image Representation,” IEEE Trans. Image Processing,    vol. 6, pp. 1213-1230, September 1997.-   [20] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri,    “Analytic banded approximation for the discretized free    propagator,” J. Physical Chemistry, vol. 95, no. 21, pp. 8299-8305,    1991.-   [21] D. K. Hoffman, A. Frishman, and D. J. Kouri, “Distributed    approximating functional approach to fitting multi-dimentional    surfaces,” Chemical Physics Lett., Vol. 262, pp. 393-399, 1996.-   [22] D. K. Hoffman, G. W. Wei, D. S. Zhang, D. J. Kouri,    “Shannon-Gabor wavelet distributed approximating functional,”    Chemical Pyiscs Letters, Vol. 287, pp. 119-124, 1998.-   [23] L. C. Jain, N. M. Blachman, and P. M. Chapell, “Interference    suppression by biased nonlinearities,” IEEE Trans. IT, vol. 41, no.    2, pp. 496-507, 1995.-   [24] N. Jayant, J. Johnston, and R. Safranek, “Signal compression    based on models of human perception”, Proc. IEEE, vol. 81, no. 10,    pp. 1385˜1422, 1993.-   [25] M. A. Kon, L. A. Raphael, “Convergence rates of multi scale and    wavelet expansions I & II,” manuscript, 1998.-   [26] J. Kovacevic, and M. Vetterli, “Perfect reconstruction filter    banks with rational sampling factors,” IEEE Trans. SP, Vol. 41, No.    6, pp. 2047-2066, June 1993.-   [27] J. Kovacevic, W. Swelden, “Wavelet families of increasing order    in arbitrary dimensions,” Submitted to IEEE Trans. Image Processing,    1997.-   [28] S. Lai, X. Li, and W. F. Bischof, “On techniques for detecting    circumscribed masses in mammograms,” IEEE Trans. Med. Imag., vol. 8,    no. 4, pp. 377-386, 1989.-   [29] A. F. Laine, S. Schuler, J. Fan and W. Huda, “Mammographic    feature enhancement by multi scale analysis,” IEEE Trans. MI, vol.    13, pp. 725-740, 1994.-   [30] J. Lu, and D. M. Healy, Jr., “Contrast enhancement via multi    scale gradient transform,” preprint.-   [31] J. Lu, D. M. Healy Jr., and J. B. Weaver, “Contrast enhancement    of medical images using multi scale edge representation,” Optical    Engineering, in press.-   [32] S. Mallat, “A theory for multiresolution signal decomposition:    the wavelet representation,” IEEE Trans. PAMI, Vol. 11, No. 7, pp.    674-693, July 1989.-   [33] S. Mallat, and S. Zhong, “Characterization of Signals from    multi scale edges,” IEEE Trans. PAMI, vol. 14, no. 7, pp. 710-732,    1992.-   [34] Y. Meyer Wavelets Algorithms and Applications, SIAM Publ.,    Philadelphia 1993.-   [35] M. Nagao and T. Matsuyama, “Edge preserving smoothing,” Comput.    Graphics and Image Processing, vol. 9, no. 4, pp. 394-407, 1979.-   [36] K. Ramchandran, M. Vetterli, “Best wavelet packet bases in a    rate-distortion sense,” IEEE Trans. Image Processing, Vol. 2, No. 2,    pp. 160-175, April 1993.-   [37] K. Ramchandran, Z. Xiong, K. Asai and M. Vetterli, “Adaptive    Transforms for Image Coding Using Spatially-varying Wavelet    Packets,” IEEE Trans. Image Processing, vol. 5, pp. 1197-1204, July    1996.-   [38] O. Rioul, M. Vetterli, “Wavelet and signal processing,” IEEE    Signal Processing Mag., pp. 14-38, October 1991.-   [39] A. M. Rohaly, A. J. Ahumada, and A. B. Watson, “Object    detection in natural backgrounds predicted by discrimination    performance and models,” Vision Research, Vol. 37, pp. 3225-3235,    1997.-   [40] N. Saito, G. Beylkin, “Multi scale representations using the    auto-correlation functions of compactly supported wavelets,” IEEE    Trans. Signal Processing, Vol. 41, no. 12, pp. 3584-3590, 1993.-   [41] A. Scheer, F. R. D. Velasco, and A. Rosenfield, “Some new image    smoothing techniques,” IEEE Trans. Syst., Man. Cyber., vol. SMC-10,    no. 3, pp. 153-158, 1980.-   [42] M. J. Shensa, “The discrete wavelet transform: wedding the a    trous and Mallat algorithms”, IEEE Trans. SP, vol. 40, no. 10, pp.    2464˜2482, 1992.-   [43] Zhuoer Shi and Zheng Bao, “Group-normalized processing of    complex wavelet packets,” Science in China, Ser. E, Vol. 40, No. 1,    pp. 28˜43, February 1997.-   [44] Z. Shi, Z. Bao, “Group-normalized wavelet packet signal    processing”, Wavelet Application IV, SPIE, vol. 3078, pp. 226˜239,    1997.-   [45] Z. Shi, Z. Bao, “Fast image coding of interval interpolating    wavelets,” Wavelet Application IV, SPIE, vol. 3078, pp. 240-253,    1997.-   [46] Zhuoer Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman,    “Perceptual image processing using Gaussian-Lagrange distributed    approximating functional wavelets”, submitted to IEEE SP Letter,    1998.-   [47] Zhuoer Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, and Z. Bao,    “Lagrange wavelets for signal processing”, Submitted to IEEE Trans.    IP, 1998.-   [48] Zhuoer Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, and Z. Bao,    “Visual multiresolution color image restoration”, Submitted to IEEE    Trans. PAMI, 1998.-   [49] Zhuoer Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, “Perceptual    normalized subband image restoration”, IEEE Symposium on    Time-frequency and Time-scale Analysis, N. 144, Pittsburgh, Pa.,    Oct. 6-9, 1998.-   [50] Zhuoer Shi, Z. Bao, “Group normalized wavelet packet    transform,” IEEE Trans. CAS II, in press, 1998.-   [51] Zhuoer Shi, Z. Bao, “Slow-moving ship target extraction using    complex wavelet packet,” submitted to IEEE Trans. SP, 1998.-   [52] Zhuoer Shi, Z. Bao, L. C. Jiao, “Normalized wavelet packet    image processing,” submitted to IEEE Trans. IP, 1998.-   [53] Zhuoer Shi, Z. Bao, L. C. Jiao, “Wavelet packet based ECG    filtering,” submitted to IEEE Trans. BE, 1998.-   [54] Zhuoer Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, “Perceptual    multiresolution enhancement”, Submitted to IEEE ICASSP, 1998.-   [55] Zhuoer Shi, D. J. Kouri, D. K. Hoffman, “Mammogram Enhancement    Using Generalized Sinc Wavelets”, Submitted to IEEE Trans. MI, 1998.-   [56] W. Swelden, “The lifting scheme: a custom-design construction    of biorthogonal wavelets,” Appl. And Comput. Harmonic Anal., vol. 3,    no. 2, pp. 186˜200, 1996.-   [57] P. G. Tahoces, J. Correa, M. Souto, C. Gonzalez, L. Gomez,    and J. J. Vidal, “Enhancement of chest and breast radiographs by    automatic spatial filtering,” IEEE Trans. Med. Imag., vol. 10, no.    3, pp. 330-335, 1991.-   [58] T. D. Tran, R. Safranek, “A locally adaptive perceptual masking    threshold model for image coding,” Proc. ICASSP, 1996.-   [59] M. Unser, A. Adroubi, and M. Eden, “The L₂ polynomial spline    pyramid,” IEEE Trans. PAMI, vol. 15, no. 4, pp. 364-379, 1993.-   [60] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part I: system-theoretic fundamentals,” IEEE    Trans. SP, Vol. 43, No. 5, pp. 1090-1102, May 1995.-   [61] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part II: the FIR case, factorizations, and    biorthogonal lapped transforms,” IEEE Trans. SP, Vol. 43, No. 5, pp.    1103-1115, May 1995.-   [62] M. Vetterli, C. Herley, “Wavelet and filter banks: theory and    design,” IEEE Trans. SP, Vol. 40, No. 9, pp. 2207-2232, September    1992.-   [63] J. D. Villasenor, B. Belzer, and J. Liao, “Wavelet filter    evaluation for image processing,” IEEE Trans. IP, vol. 4, no. 8, pp    1053-1060, 1995.-   [64] A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor,    “Visibility of wavelet quantization noise,” IEEE. Trans. Image    Processing, vol. 6, pp. 1164-1175, 1997.-   [65] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,    “Lagrange distributed approximating Functionals,” Physical Review    Letters, Vol. 79, No. 5, pp. 775˜779, 1997.-   [66] G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Wavelets and    distributed approximating functionals,” submitted to Phys. Rev.    Lett.-   [67] G. W. Wei, S. C. Althorpe, D. J. Kouri, D. K. Hoffman, “An    application of distributed approximating functional-wavelets to    reactive scattering,” J. Chemical Physics, Vol. 108, No. 17, pp.    7065-7069, May 1998.-   [68] Z. Xiong, K. Ramchandran and M. T. Orchard, “Space-frequency    Quantization for Wavelet Image Coding,” IEEE Trans. Image    Processing, vol. 6, pp. 677-693, May 1997.

DUAL WINDOW SELECTIVE AVERAGING FILTER Introduction

Local averaging is one of the simplest filtering techniques. Itpreserves the mean gray level while suppressing the variability in flatregions. However, simple linear local averaging is undesirable for imagesmoothing because it is incapable of preserving image edges. Linearlocal averaging filters are essentially low pass filters, which tend toblur the edges and fine structures in the original image. In order topreserve the edges while achieving some degree of smoothing, it isdesirable to employ nonlinear filters. A number of nonlinear selectiveaveraging methods [1-8] have been devised for this purpose. The basicidea for these methods is to select only a portion of the gray levelvalues in the local window to use in a (weighted) average. In references[3-5], an alpha-trimmed mean filter (α-TMF), which uses a “medianbasket” to select a predetermined number of pixels above and below themedian pixel to the sorted pixels of the moving window, was proposed.

The values in the basket are averaged to give the α-TMF filteringoutput. An asymmetric way to select the averaging pixels whose valuesare close to that of the median pixel was presented in references [4-6]and was named the modified trimmed mean filter (MTMF) in reference [4].Recently, we developed a generalization of the α-TMF, which we denotedas GTMF [8].

It employs the same way to select the pixels from the window forinclusion in the median basket. However, the selected pixels and thecenter pixel in the window are weighted and averaged to give thefiltering output. It has been shown [8] that the GTMF performs betterthan other well known filters for the removal of either impulse noise oradditive noise.

A new nonlinear filtering technique is disclosed, called the “dualwindow selective averaging filter” (DWSAF), to remove additive noise(e.g., Gaussian noise). Assuming implicitly that the ideal image ispiecewise flat, two normal concentric moving windows and a pixelcontainer are employed to determine the values to be used in replacingthe gray level value of the center pixel I_(c). Three steps are employedin this filtering algorithm. First, the GTMF is implemented within thesmaller window W_(S) to give an intermediate output G_(c) at the centerpixel of the window. Second, only the gray level values close to G_(c)are selected from the larger window W_(L) and put to the container C.Third, if the number of pixels in C equals zero, the final DWSAF outputD_(c) is just G_(c), otherwise, the gray level values in C are averagedto give the final DWSAF output D_(c).

In contrast to the α-TMF, the GTMF also includes the center pixel I_(c)in the averaging operation and its weight is usually larger than thoseof other pixels in the median basket, which is important for the removalof additive noise.

A threshold T based on the GTMF output G_(c) is used to select thepixels for the container. A pixel is selected for inclusion in thecontainer if its value is in the range of [G_(c)−T, G_(c)+T]. When T=0,the DWSAF is equivalent to the GTMF within the smaller window, and whenT=255 (for an 8bpp gray scale image), it becomes a simple moving averagewithin the larger window.

GENERALIZED ALPHA-TRIMMED MEAN FILTER

The implementation of the generalized trimmed mean filter (GTMF) hasbeen described in detail in [17]. The pixels {I₁, I₂, . . . , I_(m−1),I_(m), I_(m+1), . . . I_(n)} in the local window associated with a pixelI_(c), have been arranged in an ascending (or descending) order, withI_(m) being the median pixel. The key generalization to median filteringintroduced in the alpha-trimmed mean filter (α-TMF) [18] is to design amedian basket to combine a group of pixels whose gray level values areclose to the median value of the window.

An averaging operation is then utilized to generate an adjustedreplacement A_(c) for I_(c). For example, a 3-entry median basket α-TMFis implemented according to $\begin{matrix}{A_{c} = \frac{( {I_{m - 1} + I_{m} + I_{m + 1}} )}{3}} & (81)\end{matrix}$

It is evident that a single-entry median basket α-TMF is equivalent tothe median filter and a n-entry median basket α-TMF is equivalent to thesimple moving average filter.

In general, the α-TMF outperforms the median filter. However, it isstill not optimal when filtering either the additive noise corruptedimages or highly impulse noise corrupted images. For example, whenremoving additive noise, the α-TMF does not take the I_(c) as a specialpixel. As is well known, for an image corrupted by additive noise, I_(c)has the largest probability of being the closest to the true value amongall the pixels in the window. Neglecting the influence of the centerpixel is a mistake if one desires to filter additive noise. Anotherdifficulty for the α-TMF, when it is used to remove impulse noise fromhighly corrupted images, is that the pixels selected for the medianbasket may also be corrupted by impulse noise. It is thereforeunreasonable for all the pixels in the basket to have the same weight.

The GTMF uses a median basket to collect a group of pixels from thesorted pixel values of the window associated with I_(c), in the same wayas the α-TMF.

The values of the selected pixels and I_(c) are then weighted andaveraged to give the GTMF output.

For a 3-entry median basket, the output of the GTMF is given by$\begin{matrix}{G_{c} = \frac{( {{w_{1}I_{m - 1}} + {w_{2}I_{m}} + {w_{3}I_{m + 1}} + {w_{c}I_{c}}} )}{( {w_{1} + w_{2} + w_{3} + w_{c}} )}} & (82)\end{matrix}$where G_(c) is the GTMF output, w₁, w₂, w₃, and w_(c) are the weights.It is interesting to see that when w_(c)=0 and w₁=w₂=w₃≠0, the GTMFreduces to the α-TMF. When w₁=w₂=w₃=0, it becomes the standard medianfilter. Since the GTMF takes the center pixel I_(c) into account, weexpect that it should also work better for removing additive noise.

DUAL WINDOW SELECTIVE AVERAGING FILTER

In this section, we will discuss how to implement our dual windowselective averaging filter (DWSAF), based on the GTMF. The standardselective averaging algorithm [2] is not optimal because it onlycomputes the average (or weighted-average) of the gray level valuesclose to that of the center pixel in a window. However, sometimes it isunreasonable to choose the gray level value of the center pixel as acriteria because the pixel may be highly corrupted. Using the standardselective averaging algorithm to treat such a pixel will result in itremaining highly corrupted.

The key idea of the DWSAF is to find an alternative criteria for thestandard selective averaging algorithm to use. In reference [4], amodified trimmed mean filter (MTMF) was proposed to average only thosepixels whose gray level values fall within the range [M_(c)−q, M_(c)+q],where M_(c) is the median value (M_(c)=I_(m)) and q is a preselectedthreshold.

Here, we propose a new filtering algorithm which employs two differentsizes of concentric moving windows W_(S) and W_(L), and a pixelcontainer C. Three steps are involved in the filtering technique. First,the GTMF is employed in the smaller window W_(S) according to Equation(82) to give an intermediate output G_(c), which is used as the criteriafor the standard selective averaging algorithm.

Second, from the larger window W_(L), only the pixels with gray levelvalues close to G_(c) are selected and placed in a container C. In doingso, a switching threshold T is employed to determine if a pixel in W_(L)is close enough to G_(c). We determine if a pixel I_(k) in W_(L) belongsto C as follows:I _(k) ∈C, if |I _(k) −G _(c·) |<T,  (83)where T is the threshold.

Third, if the number of pixels N in the container C is greater thanzero, the gray level values of the pixels in the container are averagedwith the same weight for all pixels to give the final output of theDWSAF. If N=0, the final output equals G_(c·). We summarize this as$\begin{matrix}{D_{c} = \{ \begin{matrix}{A_{c},} & {{{if}\quad N} \neq 0} \\{G_{c},} & {{{if}\quad N} = 0}\end{matrix} } & (84)\end{matrix}$where D_(c) is the output of the DWSAF to replace I_(c) and A_(c) is theaverage of the gray level values in the container. The reason foremploying two windows is as follows. As in the case of median filter,the implementation of the GTMF using a larger window blurs the imagemore than using a smaller window. However, the output of the GTMF in asmaller window can be used as a criteria to improve the filtering resultby use of the standard selective averaging algorithm in the largerwindow.

This reduces the image blurring because we only average those gray levelvalues that are close to the GTMF output. For different choices of thethreshold T, the DWSAF can be made to be either a GTMF with respect tothe smaller window W_(S) (T=0) or a simple moving average within thelarger window W_(L) (T=255 for an 8bpp gray-scale image).

The DWSAF algorithm can be employed iteratively, which generallyimproves its performance. If the weight w_(c) of the center pixel is toohigh, the output values at some pixels may remain close to theiroriginal input values after a number of iterations, and thus someisolated impulse-like noise may be present. Numerical experimentationshows that changing the threshold T to zero after a number of iterationsgreatly alleviates this problem.

NUMERICAL EXPERIMENTS

Two numerical examples are presented here for testing our filteringalgorithm. The first one is a standard one-dimensional blocked signal,which is shown in FIG. 32(a). We degrade it with Gaussian noise of meansquare error MSE=1.00 and mean average error MAE=0.80 (see FIG. 33(b)).The sizes of the two moving windows we used are 7 and 19 for W_(S) andW_(L) respectively. The weights for the 3-entry basket which we used forthe GTMF are w₁=w₃=0, w₂=3 and w_(c)=4. The switching threshold ischosen to be T=1.5. To obtain improved filtering results, the DWSAFalgorithm is implemented recursively.

In FIG. 32(c), we show the filtered signal. It is seen from FIG. 32(c)that the filtered result is in excellent agreement with the originalnoise-free signal. The MSE and MAE of our filtered result are 6.19E-2and 6.49E-3 respectively! The DWSAF is extremely efficient and robustfor removing additive noise from blocked signals.

To further confirm the usefulness of the algorithm, we next consider atwo-dimensional benchmark 8 bit, 512 \times 512, “Lena” image, which hasbeen degraded with two different amounts of Gaussian noise. FIG. 33(a)shows the corrupted Lena image with peak signal-to-noise ratioPSNR=22.17 dB and FIG. 34(a) shows the corrupted image with PSNR=18.82dB. Since the original noise-free image is not exactly piecewise flat,it is not desirable to produce a filtering performance as efficient asin the first example. However, we expect that the DWSAF will yieldimproved results.

The sizes of the two windows for both corrupted images are 3 for W_(S)and 5 for W_(L). Three-entry median basket is used for both corruptedimages, and the weights are w₁:w₂:w₃:w_(c)=1:1:1:10 for the image shownin FIG. 33(a) and w₁:w₂:w₃:w_(c)=1:1:1:50 for the image shown in FIG.34(a). The initial switching threshold T is 50 for both corruptedimages, and is changed to 0 after the first iteration for the lowernoise image and after the second iteration for the higher noise image.FIGS. 33(b) and 34(b) show the filtered images produced by our numericalalgorithm. Comparing them with the degraded images, it is clear that thealgorithm can effectively remove noise and significantly preserve edgessimultaneously. For a quantitative evaluation of the performance of thealgorithm, the PSNR, mean square error (MSE) and mean absolute error(MAE) comparison for different filtering algorithms are listed in TABLE2.

TABLE 2 Comparative Filtering Results for Lena Image Corrupted withDifferent Amount of Gaussian Noise Noisy Image I Noisy Image IIAlgorithm* PSNR MSE MAE PSNR MSE MAE No Denoising 22.17 dB 394.37 15.9718.82 dB 853.07 23.71 Median (3 × 3) 29.38 dB 75.09 6.26 27.60 dB 112.977.72 Median (5 × 5) 28.60 dB 89.66 6.53 27.39 dB 118.41 7.53 α-TMP (3 ×3) 29.84 dB 67.46 6.07 28.13 dB 100.10 7.04 MTMF (3 × 3) 29.91 dB 66.465.79 28.23 dB 97.74 7.39 New Approach 30.69 dB 55.50 5.34 28.96 dB 82.696.45 *All results are implemented recursively for optimal PSNRperformance. The threshold used for the MTMF algorithm is optimized to q= 65 for image I and q = 100 for image II.

The PSNR is increased about 8.52 dB for the lower noise image and 10.14dB for the higher noise image, both of which are better than the bestmedian, α-TMF and MTMF filtering results.

CONCLUSIONS

In this disclosure, a new filtering algorithm for removing additivenoise from a corrupted image is presented. In comparison to standardfiltering schemes, the DWSAF algorithm employs two concentric movingwindows to determine the gray level value used to replace that of thecenter pixel. Employing the larger window does not lead to significantblurring of the image because of the effect of the smaller window. TheGTMF is used in the smaller window for obtaining the criterion forstandard selective averaging in the larger window. Numerical examplesshow that the proposed filtering algorithm is both efficient and robustfor recovering blocked signals contaminated with additive noise.

REFERENCES

-   [1] A. Scher, F. R. D. Velasco, and A. Rosenfeld, “Some new image    smoothing techniques,” IEEE. Trans. Syst. Man, Cybern., Vol. SMC-10,    pp. 153-158, 1980.-   [2] A. Rosenfeld and A. C. Kak, Digital Picture Processing, New    York: Academic Press, 1982, Vol. 1.-   [3] J. B. Bednar and T. L. Watt, “Alpha-trimmed means and their    relationship to median filters,” IEEE Trans. Acoust., Speech, and    Signal Processing, Vol. ASSP-32, pp. 145-153, 1984.-   [4] Y. H. Lee, S. A. Kassam, “Generalized median filtering and    related nonlinear filtering techniques,” IEEE Trans. Acoust.,    Speech, and Signal Processing, Vol. ASSP-33, pp. 672-683, 1985.-   [5] S. R. Peterson, Y. H. Lee, and S. A. Kassam, “Some statistical    properties of alpha-trimmed mean and standard type M filters,” IEEE    Trans. Acoust., Speech, and Signal Processing, Vol. ASSP-36, pp.    707-713, 1988.-   [6] P. K. Sinha and Q. H. Hong, “An improved median filter”, IEEE    Trans. Medical Imaging, Vol. 9, pp. 345-346, 1990.-   [7] X. You and G. Grebbin, “A robust adaptive estimator for    filtering noise in images”, IEEE Trans. Image Processing, Vol. 4,    pp. 693-699, 1995.-   [8] D. S. Zhang, Z. Shi, D. J. Kouri, D. K. Hoffman, “A new    nonlinear image filtering technique,” Optical Engr., Submitted.

ARBITRARY DIMENSION EXTENSION OF SYMMETRIC DAF Introduction

The distributed approximating functional (DAF) approach to the timeevolution of wave packets is a powerful, new method aimed at takingadvantage of the local nature of the potential in the coordinaterepresentation for ordinary chemical collisions, and the localizednature of the kinetic energy in this same representation. These featuresof the Hamiltonian lead to a highly band-pass expression for thepropagator, specific to a restricted class of wave packets (DAFs). Withcontrol of the parameters which determine it, the DAF class can be madeto include the wave packets of interest in any particular time-dependentquantum problem. Because the propagator is banded, its application tothe propagation on the grids (1) scales like the number of grid points,N, in any dimension (which is the ultimate for any grid methods; thescaling constant depends on the band width); (2) requires reducescommunication time when implemented on massively parallel computer; and(3) minimizes the storage requirement for the propagator.

In general, DAF can be regarded as a radial interpolating basis, as wellas the scaling function. The corresponding DAF wavelets can beimplemented using generalized DAFs. As a kind of radial basis functional(RBF), DAF neural networks are constructed and applied in signalprocessing.

We presented the derivation of the DAFs independent of the choice ofcoordinates and show how the Hermite functions becomes generalized forarbitrary numbers of degrees of freedom and choice fo orthogonalcoordinates. We discuss the properties of the generalized “DAFpolymonals” associate with different coordinate and numbers of degreesof freedom, and how angular momentum conservation leads to a radialpropagator for spherical waves. Symmetric quincunx-DAF Dirichlet classesare carefully designed for hyper spherical surface representation.

DAF THEORY

The theory of fundamental distributed approximating functional is basedon the Hermite function representation of the 1D δ function which is$\begin{matrix}{{\delta( {x❘\sigma} )} = {\frac{1}{\sigma\sqrt{2\pi}}\quad\exp\quad( \frac{- x^{2}}{2\sigma^{2}} ){\sum\limits_{n = 0}^{\infty}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{n!}{H_{2n}( \frac{x}{\sqrt{2\sigma}} )}}}}} & (85)\end{matrix}$

The function H_(2n) is the Hermite polynomial of even order, 2n. Thequantity σ is the width of the Gaussian window of the Hermitepolynomial. The qualitative behavior of one particular Hermite DAF isshown in FIG. 1(a). The Hermite polynomial H is generated by the usualrecursion $\begin{matrix}{{H_{n}(x)} = \{ \quad\begin{matrix}{1,} & {n = 0} \\{{2x},} & {n = 1} \\{{{2{{xH}_{n - 1}(x)}} - {2( {n - 1} ){H_{n - 2}(x)}}},} & {n > 1}\end{matrix}\quad } & (86)\end{matrix}$

The predominant advantage of the Hermite polynomial approximation is itshigh-order derivative preservation (which leads to a smoothapproximation). This disclosure expands and tests the approach of Madanand Milne (1994) for pricing contingent claims as elements of aseparable Hilbert space. We specialize the Hilbert space basis to thefamily of Hermite polynomials and use the model to price options onEurodollar futures. Restrictions on the prices of Hermite polynomialrisk for contingent claims with different times to maturity are derived.These restrictions are rejected by our empirical tests of afour-parameter model. The unrestricted results indicate skewness andexcess kurtosis in the implied risk-neutral density. Thesecharacteristics of the density are also mirrored in the statisticaldensity estimated from a time series on LIBOR. The out-of-sampleperformance of the four-parameter model is consistently better than thatof a two-parameter version of the model.

The Hermite polynomial are given in terms of their generator by$\begin{matrix}{{{H_{n}(x)}{\exp( {- x^{2}} )}} = {{( {- 1} )^{n}\frac{\mathbb{d}^{n}}{\mathbb{d}x^{n}}{\exp( {- x^{2}} )}} = {( {- 1} )^{n}{\nabla_{x}^{n}{\exp( {- x^{2}} )}}}}} & (87)\end{matrix}$where ∇_(x) is the x component of the gradient operator. This equationprovides two equivalent and useful ways in which the above function canbe expressed as $\begin{matrix}{{\delta( {x - x^{\prime}} \middle| \sigma )} = {\frac{1}{\sigma\sqrt{2\pi}}{\sum\limits_{n = 0}^{\infty}\quad{( {- \frac{1}{4}} )^{n}{\frac{1}{n!}\lbrack {2\sigma^{2}} \rbrack}^{n}{\nabla_{x^{\prime}}^{2n}{\exp\lbrack {{{- ( {x - x^{\prime}} )^{2}}/2}\sigma^{2}} \rbrack}}}}}} & (88)\end{matrix}$or $\begin{matrix}{{\delta( {x - x^{\prime}} \middle| \sigma )} = {\frac{1}{\sigma\sqrt{2\pi}}{\sum\limits_{n = 0}^{\infty}\quad{( {- \frac{1}{4}} )^{n}{\frac{1}{n!}\lbrack {2\sigma^{2}} \rbrack}^{n}{\nabla_{x^{\prime}}^{2n}{\exp\lbrack {{{- ( {x - x^{\prime}} )^{2}}/2}\sigma^{2}} \rbrack}}}}}} & (89)\end{matrix}$where in the second equation ∇_(x) ^(2n) has been replaced by ∇_(x′)^(2n). The two are equivalent because the derivative acts on an evenfunction of (x−x′).

The simple practical implementation is truncating the sum at somemaximum value M/2 to obtain $\begin{matrix}{{\delta_{M}( {x - x^{\prime}} \middle| \sigma )} = {\frac{1}{\sigma\sqrt{2\pi}}{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}{\frac{1}{n!}\lbrack {2\sigma^{2}} \rbrack}^{n}{\nabla_{x}^{2n}{\exp\lbrack {{{- ( {x - x^{\prime}} )^{2}}/2}\sigma^{2}} \rbrack}}}}}} & (90)\end{matrix}$as indicated by the notation, the distributed approximating functionalδ_(M)(x|σ) depend upon the M and σ, nut it does not depend upon thecoordinate system per se. Here it is easy to show that $\begin{matrix}{{\lim\limits_{Marrow\infty}{\delta_{M}( {x - x^{\prime}} \middle| \sigma )}} = {\delta( {x - x^{\prime}} )}} & (91)\end{matrix}$for any fixed M. The availability of two independent parameters, eitherof which can be used to generate the identity kernel or Dirac deltafunction, can be viewed as the source of robustness of the DAFs ascomputational tools. See e.g., D. K. Hoffman, D. J. Kouri, “Distributedapproximating functional theory for an arbitrary number of particles ina coordinate system-independent formalism,” J. Physical Chemistry,Vol.97, No.19, pp.4984-4988, 1993. A simple expression of Hermite DAF isas $\begin{matrix}{{\delta_{M}( x \middle| \sigma )} = {\frac{1}{\sigma\sqrt{2\pi}}{\exp( \frac{- x^{2}}{2\sigma^{2}} )}{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{n!}{H_{2n}( \frac{x}{\sqrt{2}\sigma} )}}}}} & (92)\end{matrix}$

ARBITRARY DAF EXTENSION IN MULTI-DIMENSION SPACE

Tensor-product extension

Different selections of interpolating shells result in different DAFs.Theoretically, this kind of functional can be regarded as the smoothingoperator or the scaling function in wavelet theory. It can be used togenerate the corresponding wavelets (differential functionals) forsignal analysis. The discrete wavelet transform is implemented usingfilterbanks. Hoffman and Kouri gave a tensor-product extension of DAF inmulti-dimensional Hilbert space. It can be explained in detail asfollowing. Consider a Dirichlet system which can be represented in ageneral configuration space by a vector R with an arbitrary number N ofCartesian components, x₁,x₂, . . . , x_(N). By “Cartesian” we mean thatthe “tensor” volume element for the space is given by $\begin{matrix}{{\mathbb{d}X} = {\prod\limits_{j = 1}^{N}{\mathbb{d}x_{j}}}} & (93)\end{matrix}$Thus, an arbitrary function, Ψ(X), in Hibert space can be represented byΨ(X)=∫δ(X−X′) Ψ(X′)dX′  (94)where δ(X−X′) is the δ function in N-dimension space which can bewritten in the form $\begin{matrix}{{\delta( {X - X^{\prime}} )} = {\prod\limits_{j = 1}^{N}{\delta( {x_{j} - x_{j}^{\prime}} )}}} & (95)\end{matrix}$Using the tensor product to substitute the 1D scalar variable, we canobtain the expression in N-dimensional Hilbert space as $\begin{matrix}{{\delta( {{X - X^{\prime}}❘\sigma} )} = {( {\sigma\sqrt{2\quad\pi}} )^{- N}{\sum\limits_{n = 0}^{\infty}{( {- \frac{1}{4}} )^{n}\quad{\frac{1}{n!}\lbrack {2\sigma^{2}} \rbrack}^{n}{\nabla_{X}^{2n}{\exp\lbrack {{{- ( {X - X^{\prime}} )^{2}}/2}\sigma^{2}} \rbrack}}}}}} & (96)\end{matrix}$where the expression $\begin{matrix}{{\sum\limits_{n = 0}^{\infty}( \cdot )} = {\sum\limits_{n_{1} = 0}^{\infty}{\sum\limits_{n_{2} = 0}^{\infty}\quad{\ldots\quad{{\sum\limits_{n}}_{N}^{\infty}\quad( \cdot )}}}}} & (97)\end{matrix}$and the multi-variable $\begin{matrix}{n!={\prod\limits_{j = 1}^{N}{n_{j}!}}} & (98)\end{matrix}$In multi-dimensional Hilbert space $\begin{matrix}{n = {\sum\limits_{j = 1}^{N}n_{j}}} & (99)\end{matrix}$The gradient operation is $\begin{matrix}{\nabla_{x}^{2n}{= {\frac{\mathbb{d}^{n_{1}}}{\mathbb{d}x_{1}^{n_{1}}}\quad\frac{\mathbb{d}^{n_{2}}}{\mathbb{d}x_{2}^{n_{2}}}\quad\ldots\quad\frac{\mathbb{d}^{n_{N}}}{\mathbb{d}x_{N}^{n_{N}}}}}} & (100)\end{matrix}$where ∇_(x) ² is the N-dimensional Laplacian operator. $\begin{matrix}{( {X - X^{\prime}} )^{2} = {\sum\limits_{j = 1}^{N}( {x_{j} - x_{j}^{\prime}} )^{2}}} & (101)\end{matrix}$

The advantage of this expression is that it is independent of theparticular choice of coordinate system and we can use whatevercoordinate are convenient for a particular problem (e.g. Cartesian,cylindrical polar, spherical polar, hyperspherical, elliptical, etc., asthe case might be). The generalized interpolating matrix is hyper-squareand can be separated by different directional vector.

Tensor-product extension

DAF define a class of symmetric interpolants. Besides the tensor-productextension, it could be with more interesting shapes. To preserve thesymmetry property, any DAF or DAF-like inerpolaitng functional isexpressed asδ(x)=δ(|x|)  (102)

Therefore the fundamental variable for DAF processing is |x|. Theextended norm-preserved quincunx solution is implemented by thefollowing substitution in N-D space $\begin{matrix}{{X} = ( {\sum\limits_{j = 1}^{N}{x_{j}}^{p}} )^{1/q}} & (103)\end{matrix}$As p=q, the N-D space multi-variable becomes to the 1^(p) norm$\begin{matrix}{{X}_{p} = ( {\sum\limits_{j = 1}^{N}{x_{j}}^{p}} )^{1/p}} & (104)\end{matrix}$When p equals to ½, the contour of the 2D quincunx DAF is apseudo-diamond shape.

The practical truncated shape is represented by the following formula:w(x,y)=e ^(−(|x|−|y|))  (105)w(x,y)=e ^(−(|x|+|y|)) ² ^(/(2σ) ² ⁾  (106) w(x,y)=e ^(−(x+y)) ² ^(/(2σ) ² ⁾  (107)$\begin{matrix}{{w( {x,y} )} = {\mathbb{e}}^{{{- \sqrt{x^{2} + y^{2}}}/{(\quad}}2\quad\sigma^{2}{\quad)}}} & (108)\end{matrix}$

CONCLUSION

A new algorithm for impulse noise removal was presented. A group ofsignificant values in the neighboring window of one pixel are bundledand weighted to obtain a modified luminance estimation (MLE). Athreshold selective-pass technique is employed to determine whether anygiven pixel should be replaced by its MLE. Iterative processing improvesthe performance of our algorithm for highly corrupted images. Numericalexperiments show that our technique is extremely robust and efficient toimplement, and leads to significant improvement over other well-knownmethods.

LAGRANGE WAVELETS FOR SIGNAL PROCESSING Introduction

The theory of interpolating wavelets based on a subdivision scheme hasattracted much attention recently. It possesses the attractivecharacteristic that the wavelet coefficients are obtained from linearcombinations of discrete samples rather than from traditional innerproduct integrals. Mathematically, various interpolating wavelets can beformulated in a biorthogonal setting. Harten has described a kind ofpiecewise biorthogonal wavelet construction method [12]. Sweldenindependently has developed essentially this method into the well known“lifting scheme” theory [32], which can be regarded as a special case ofthe Neville filters [19]. Unlike the previous method for constructingbiorthogonal wavelets, which relies on explicit solution of coupledalgebraic equations [10], the lifting scheme enables one to construct acustom-designed biorthogonal wavelet transforms assuming only a singlelow-pass filter without iterations.

Generally speaking, the lifting-interpolating wavelet theory is closelyrelated to: the finite element technique for the numerical solution ofpartial differential equations, the subdivision scheme for interpolationand approximation, multi-grid generation and surface fitting techniques.The most attractive feature of the approach is that discrete samplingsare made which all identical to wavelet multiresolution analysis.Without any of the pre-conditioning or post-conditioning processesrequired for accurate wavelet analysis, the interpolating waveletcoefficients can be implemented using a parallel computational scheme.

Lagrange interpolation polynomials are commonly used for signalapproximation and smoothing, etc. By carefully designing theinterpolating Lagrange functionals, one can obtain smooth interpolatingscaling functions with arbitrary order of regularity. In thisdisclosure, we will present three different kinds of biorthogonalinterpolating Lagrange wavelets (Halfband Lagrange wavelets, B-splineLagrange wavelets and Gaussian-Lagrange DAF wavelets) as specificexamples of generalized interpolating Lagrange wavelets.

Halfband Lagrange wavelets can be regarded as an extension of Dubucinterpolating functionals [8, 11], auto-correlation shell waveletanalysis [26], and halfband filters [1]. B-spline Lagrange Wavelets aregenerated by a B-spline-windowed Lagrange functional which increases thesmoothness and localization properties of the simple Lagrange scalingfunction and related wavelets.

Lagrange Distributed Approximating Functionals (LDAF)-Gaussian modulatedLagrange polynomials have been successfully applied for numericallysolving various linear and nonlinear partial differential equations[40]. Typical examples include DAF-simulations of 3-dimensional reactivequantum scattering and 2-dimensional Navier-Stokes fluid flow withnon-periodic boundary conditions. In terms of wavelet analysis, DAFs canbe regarded as particular scaling functions (wavelet-DAFs); theassociated DAF-wavelets can be generated in a number of ways [41]. BothDAFs and DAF-wavelets are smooth and decay rapidly in both the time andfrequency representations. One objective of the present work is toextend the DAF approach to signal and image processing by constructingnew biorthogonal DAF-wavelets and their associated DAF-filters using thelifting scheme [32].

As an example application of Lagrange wavelets, we consider imageprocessing, de-noising and restoration. This application requiresdealing with huge data sets, complicated space-frequency distributionsand complex perceptual dependent characteristics. De-noising andrestoration play an important role in image processing. Noise distortionnot only affects the visual quality of images, but also degrades theefficiency of data compression and coding.

To explicit the time-frequency characteristics of wavelets, an earliergroup normalization (GN) technique [28, 29] has been utilized tore-scale the magnitudes of various subband filters and obtain normalizedequivalent decomposition filters (EDFs). The group normalization processcorrects the drawback that the magnitudes of the transform coefficientsdo not correctly reflect the true strength of the various signalcomponents.

Moreover, in order to achieve the best noise-removing efficiency, thehuman vision response is best accounted for by a perceptualnormalization (PN) based on the response property of the Human VisionSystem (HVS). The concept of visual loss-less quantization, introducedby Watson [39], is utilized to construct the visual loss-less matrix,which modifies the magnitudes of the normalized wavelet coefficients.

Perceptual signal processing has the potential of overcoming the limitsof the traditional Shannon Rate-distortion (R-D) theory forperception-dependent information, such as images and acoustic signals.Previously, Ramchandran, Vetterli, Xiong, Herley, Asai, and Orchard haveutilized a rate-distortion compromise for image compression [14, 23, 24,and 42]. Our recently derived Visual Group Normalization (VGN) technique[31] can likely be used with rate-distortion compromise to generate aso-called Visual Rate-Distortion (VR-D) theory to further improve imageprocessing.

Softer Logic Masking (SLM) is an adjusted de-noising technique [29],designed to improve the filtering performance of Donoho's Soft Threshold(ST) method [9]. The SLM technique efficiently preserves importantinformation, particularly at an edge transition, in a mannerparticularly suited to human visual perception.

INTERPOLATING WAVELETS

The basic characteristics of interpolating wavelets of order D discussedin reference [10] require that the primary scaling function, φ,satisfies the following conditions.

(1) Interpolation: $\begin{matrix}{{\phi(k)} = \{ \quad{{\begin{matrix}{1,} & {k = 0} \\{0,} & {k \neq 0}\end{matrix}\quad k} \in Z}\quad } & (109)\end{matrix}$

(2) Self-induced Two-Scale Relation: φ can be represented as a linearcombination of dilates and translates of itself, while the weight is thevalue of φ at a subdivision integer of order 2. $\begin{matrix}{{\phi(x)} = {\sum\limits_{k}{{\phi( {k/2} )}{\phi( {{2x} - k} )}}}} & (110)\end{matrix}$This is only approximately satisfied for some of the interpolatingwavelets discussed in the later sections; however, the approximation canbe made arbitrarily accurate.

(3) Polynomial Span: For an integer D≧0, the collection of formal sums,symbolized by ΣC_(k)φ(x−k), contains all polynomials of degree D.

(4) Regularity: For real V>0, φ is Hölder continuous of order V.

(5) Localization: φ and all its derivatives through order └V┘ decayrapidly.

 |φ^((r))(x)|≦A _(s)(1|x|)^(−s) , x∈R, s>0, 0≦r≦└V┘  (111)

where └V┘ represents the maximum integer which does not exceed V.

Interpolating wavelets are particularly efficient for signalrepresentation since their multiresolution analysis can be simplyrealized by discrete sampling. This makes it easy to generate a subbanddecomposition of the signal without requiring tedious iterations.Moreover, adaptive boundary treatments and non-uniform samplings can beeasily implemented using interpolating methods.

Compared with commonly used wavelet transforms, the interpolatingwavelet transform possesses the following characteristics:

1. The wavelet transform coefficients are generated by linearcombination of signal samplings, instead of the commonly usedconvolution of wavelet transform, such as $\begin{matrix}{W_{j,k} = {\int_{R}{{\psi_{j,k}(x)}\quad f\quad(x)\quad{\mathbb{d}x}}}} & (112)\end{matrix}$where ψ_(j,k)(x)=2^(j/2)ψ(2^(j)x−k).

2. A parallel-computing algorithm can be easily constructed. Thecalculation and compression of coefficients are not coupled. For thehalfband filter with length N, the calculation of the waveletcoefficients, W_(j,k), does not exceed N+2 multiply/adds for each.

3. For a D-th order differentiable function, the wavelet coefficientsdecay rapidly.

4. In a mini-max sense, threshold masking and quantization are nearlyoptimal for a wide variety of regularization algorithms.

Theoretically, interpolating wavelets are closely related to thefollowing wavelet types:

Band-limit Shannon wavelets

The π band-limited function, (x)=sin(x)/(x)C in Paley-Wiener space,constructs interpolating functions. Every band-limited function ƒL²(R)can be reconstructed by the equation $\begin{matrix}{{f(x)} = {\sum\limits_{k}{{f(k)}\quad\frac{\sin\quad{\pi( {x - k} )}}{\pi( {x - k} )}}}} & (113)\end{matrix}$where the related wavelet function (the Sinclet) is defined as (see FIG.35) $\begin{matrix}{{\psi(x)} = \frac{{\sin\quad\pi\quad( {{2x} - 1} )} - {\sin\quad\pi\quad( {x - {1/2}} )}}{\pi\quad( {x - {1/2}} )}} & (114)\end{matrix}$

Interpolating fundamental spline

The fundamental polynomial spline of degree D, η^(D)(x), where D is anodd integer, has been shown by Schoenberg (1972) to be an interpolatingwavelet (see FIG. 36). It is smooth with order R=D−1, and itsderivatives through order D−1 decay exponentially [34]. Thus$\begin{matrix}{{\eta^{D}(x)} = {\sum\limits_{k}{{\alpha^{D}(k)}{\beta^{D}( {x - k} )}}}} & (115)\end{matrix}$where β^(D)(x) is the B-spline of order D defined as $\begin{matrix}{{\beta^{D}(x)} = {\sum\limits_{j = 0}^{D + 1}{\frac{( {- 1} )^{j}}{D!}\begin{pmatrix}{D + 1} \\j\end{pmatrix}( {x + \frac{D + 1}{2} - j} )^{D}{U( {x + \frac{D + 1}{2} - j} )}}}} & (116)\end{matrix}$Here U is the step function $\begin{matrix}{{U(x)} = \{ \begin{matrix}{0,\quad{x < 0}} \\{1,\quad{x \geq 0}}\end{matrix} } & (117)\end{matrix}$and {α^(D)(k)} is the sequence that satisfies the infinite summationcondition $\begin{matrix}{{\sum\limits_{k}{{\alpha^{D}(k)}{\beta^{D}( {n - k} )}}} = {\delta(n)}} & (118)\end{matrix}$Deslauriers-Dubuc functional

Let D be an odd integer, and D>0. There exist functions, F_(D), suchthat if F_(D) has already been defined at all binary rationals withdenominator 2^(j), it can be extended by polynomial interpolation, toall binary rationals with denominator 2^(j+1), i.e. all points halfwaybetween previously defined points [8, 11]. Specially, to define thefunction at (k+½)/2^(j) when it is already defined at all {k2^(−j)}, fita polynomial π_(j,k) to the data (k′/2^(j)), F_(D)(k′/2^(j)) fork′∈{2^(−j)[k−(D−1)/2], . . . , 2^(−j)[k+(D+1)/2]}. This polynomial isunique $\begin{matrix}{{F_{D}( \frac{k + {1/2}}{2^{j}} )} \equiv {\pi_{j,k}( \frac{k + {1/2}}{2^{j}} )}} & (119)\end{matrix}$

This subdivision scheme defines a function that is uniformly continuousat the rationals and has a unique continuous extension. The functionF_(D) is a compactly supported interval polynomial and is regular; it isthe auto-correlation function of the Daubechies wavelet of order D+1.This function is at least as smooth as the corresponding Daubechieswavelets.

Auto-correlation shell of orthonormal wavelets

If {hacek over (φ)} is an orthonormal scaling function, itsauto-correlation φ(x)=∫{hacek over (φ)}(t)*{hacek over (φ)}(x−t)dt is aninterpolating wavelet (FIG. 37) [26]. Its smoothness, localization andtwo-scale relations derive from {hacek over (φ)}. The auto-correlationof Haar, Lamarie-Battle, Meyer, and Daubechies wavelets lead,respectively, to the interpolating Schauder, interpolating spline, C^(∞)interpolating, and Deslauriers-Dubuc wavelets.

Lagrange half-band filters

Ansari, Guillemot, and Kaiser [1] have used Lagrange symmetric halfbandFIR filters to design the orthonormal wavelets that express the relationbetween the Lagrange interpolators and Daubechies wavelets [6]. Theirfilter corresponds to the Deslauriers-Dubuc wavelet of order D=7 (2M−1),M=4. The transfer function of the halfband symmetric filter h is givenbyH(z)=½+zT(z ²)  (120)where T is a trigonometric polynomial. Except for h(0)=½, at every eveninteger lattice h(2n)=0, n≠0, n∈2. The transfer function of thesymmetric FIR filter h(n)=h(−n), has the form $\begin{matrix}{{H(z)} = {{1/2} + {\sum\limits_{n = 1}^{M}{{h( {{2n} - 1} )}\quad( {z^{1 - {2n}} + z^{{2n} - t}} )}}}} & (121)\end{matrix}$

The concept of an interpolating wavelet decomposition is similar to thatof “algorithm a trous,” the connection having been found by Shensa [27].The self-induced scaling conditions and interpolation condition are themost important characteristics of interpolating wavelets. According tothe following equation $\begin{matrix}{{f(x)} = {\sum\limits_{n}{{f(n)}\quad\phi\quad( {x - n} )}}} & (122)\end{matrix}$the signal approximation is exact on the discrete sampling points, whichdoes not hold in general for commonly used non-interpolating wavelets.

LAGRANGE WAVELETS

Halfband Lagrange Wavelets

The halfband filter is defined as whose even samples of the impulseresponse are constrained such as h(0)=½ and h(2n)=0 for n=±1, ±2, . . .. A special case of symmetric halfband filters can be obtained bychoosing the filter coefficients according to the Lagrange interpolationformula. The filter coefficients are then given by $\begin{matrix}{{h( {{2n} - 1} )} = \frac{( {- 1} )^{n + M - 1}{\prod\limits_{m = 1}^{2M}( {M + {1/2} - m} )}}{{( {M - n} )!}\quad{( {M + n - 1} )!}\quad( {{2n} - 1} )}} & (123)\end{matrix}$These filters have the property of maximal flatness. They possess abalance between the degree of flatness at zero frequency and flatness atthe Nyquist frequency (half sampling).

These half-band filters can be utilized to generate the interpolatingwavelet decomposition, which can be regarded as a class ofauto-correlated shell orthogonal wavelets such as the Daubechieswavelets [6]. The interpolating wavelet transform can also be generatedby different Lagrange polynomials, such as [26] according to$\begin{matrix}{{P_{{2n} - 1}(x)} = {\prod\limits_{{m = {{- M} + 1}},{m \neq n}}^{M}\frac{x - ( {{2m} - 1} )}{( {{2n} - 1} ) - ( {{2m} - 1} )}}} & (124)\end{matrix}$The predicted interpolation can be expressed as $\begin{matrix}{{{\Gamma\quad{S_{j}(i)}} = {\sum\limits_{n = 1}^{M}{{P_{{2n} - 1}(0)}\lbrack {{S_{j}( {i + {2n} - 1} )} + {S_{j}( {i - {2n} + 1} )}} \rbrack}}},{i = {{2k} + 1}},} & (125)\end{matrix}$where Γ is a projection and the S_(j) are the jth layer low-passcoefficients. This projection relation is equivalent to the subbandfilter response of h(2n−1)=P _(2n−1)(0)  (126)

The above-mentioned interpolating wavelets can be regarded as anextension of the fundamental Deslauriers-Dubuc interactive sub-divisionscheme (factorized as M=2, while the order of Lagrange polynomial isD=2M−1=3) (FIG. 40(a)).

It is easy to verify that an increase of the Lagrange polynomial orderwill introduce higher regularity in the interpolating functionals (FIG.41(a)). When D→+∞, the interpolating functional becomes the-band-limited Sinc function and its domain of definition is the realline. The subband filters generated by Lagrange interpolatingfunctionals satisfy

(1) Interpolation: h(ω)+h(ω+π)=1

(2) Symmetry: h(ω)=h(−ω)

(3) Vanishing Moments: ∫_(R)x^(p)φ(x)dx=δ_(p)

Donoho outlines a basic subband extension for perfect reconstruction. Hedefines the wavelet function asψ(x)=φ(2x−1)  (127)The biorthogonal subband filters can be expressed as{tilde over (h)}(ω)=1, g(ω)=e ^(−1ω) , {tilde over (g)}(ω)=e ^(−1ω){overscore (h(ω+π))}  (128)However, the Donoho interpolating wavelets have some drawbacks, becausethe low-pass coefficients are generated by a sampling operation only asthe decomposition layer increases, the correlation between low-passcoefficients becomes weaker. The interpolating (prediction) error(high-pass coefficients) strongly increases, which destroys the compactrepresentation of the signal. Additionally, it does not lead to a Rieszbasis for L²(R) space.

Swelden has provided, by far, the most efficient and robust scheme [32]for constructing biorthogonal wavelet filters. His approach is togenerate high-order interpolating Lagrange wavelets with increasedregularity. As FIG. 38 shows, P₀ is the interpolating predictionprocess, and P₁ the so-called the updating filter, makes thedown-sampling low-pass coefficients smoother. If we choose P₀ to be thesame as P₁, then the new interpolating subband filters can be depictedas $\begin{matrix}\{ \quad\begin{matrix}{{h_{1}(\omega)} = {h(\omega)}} \\{{{\overset{\sim}{h}}_{1}(\omega)} = {1 + {{\overset{\sim}{g}(\omega)}\overset{\_}{P( {2\quad\omega} )}}}} \\{{g_{1}(\omega)} = {{\mathbb{e}}^{{- 1}\omega} - {{h(\omega)}{P( {2\quad\omega} )}}}} \\{{{\overset{\sim}{g}}_{1}(\omega)} = {\overset{\sim}{g}(\omega)}}\end{matrix}\quad  & (129)\end{matrix}$The newly developed filters h₁, g₁, {tilde over (h)}₁, and {tilde over(g)}₁ also construct the biorthogonal dual pair for perfectreconstruction. Examples of generated biorthogonal lifting wavelets withdifferent regularity are shown in FIG. 39, FIG. 40 and FIG. 41.B-Spline Lagrange Wavelets

Lagrange polynomials are natural interpolating expressions. Utilizing adifferent expression for the Lagrange polynomials, we can constructother types of interpolating wavelets.

We define a class of symmetric Lagrange interpolating functional shellsas $\begin{matrix}{{P_{M}(x)} = {\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}} & (130)\end{matrix}$It is easy to verify that this Lagrange shell also satisfies theinterpolating condition on discrete integer points, since$\begin{matrix}{{P_{M}(k)} = \{ \begin{matrix}{1,\quad{k = 0}} \\{0,\quad{otherwise}}\end{matrix} } & (131)\end{matrix}$However, simply defining the filter response ash(k)=P(k/2)/2, k=−M, M  (132)will lead to non-stable interpolating wavelets, as shown in FIG. 43.

Utilizing a smooth window, which vanishes at the zeros of the Lagrangepolynomial, will lead to more regular interpolating wavelets andequivalent subband filters (as in FIGS. 44 and 45). If we select awell-defined B-spline function as the weight window, then the scalingfunction (mother wavelet) can be defined as an interpolating B-SplineLagrange functional (BSLF) $\begin{matrix}{{\phi_{M}(x)} = {{\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}\quad{P_{M}(x)}} = {\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}{\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}}}} & (133)\end{matrix}$where N is the B-spline order, and η is the scaling factor to controlthe window width. To ensure coincidence of the zeroes of the B-splineand the Lagrange polynomial, we set 2M=η×(N+1)  (134)To preserve the interpolation condition, the B-spline envelope factor Mmust be odd number. It is easy to show that when the B-spline order isN=4k+1, η can be any odd integer (2k+1). If N is an even integer, then ηcan only be 2. When N=4k−1, we cannot construct an interpolating shellaccording to the above definition.

From the interpolation and self-induced scaling of the interpolatingwavelets, it is easy to establish thath(k)=φ_(M)(k/2)/2, k=−2M+1, 2M−1   (135)Gaussian-Lagrange DAF Wavelets

We can also select a distributed approximating functional-GaussianLagrange DAF (GLDAF) as our basic scaling function to constructinterpolating wavelets. These are $\begin{matrix}{{\phi_{M}(x)} = {{{W_{\sigma}(x)}{P_{M}(x)}} = {{W_{\sigma}(x)}{\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}}}} & (136)\end{matrix}$where W_(σ)(x) is a window function which is selected to be a Gaussian,W _(σ)(x)=e ^(−x) ² ^(/2σ) ²   (137)because it satisfies the minimum frame bound condition in quantumphysics. Here σ is a window width parameter, and P_(M)(x) is theLagrange interpolation kernel. The DAF scaling function has beensuccessfully introduced as the basis for an efficient and powerful gridmethod for quantum dynamical propagations [40]. Using the lifting scheme[32], a wavelet basis is generated. The Gaussian window in ourDAF-wavelets efficiently smooths out the Gibbs oscillations, whichplague most conventional wavelet bases. The following equation shows theconnection between the B-spline function and the Gaussian window [34]:$\begin{matrix}{{\beta^{N}(x)} \cong {\sqrt{\frac{6}{\pi( {N + 1} )}}{\exp( \frac{{- 6}x^{2}}{N + 1} )}}} & (138)\end{matrix}$for large N. As in FIG. 46, if we choose the window width to be σ=η√{square root over ((N+1)/12)},  (139)the Gaussian Lagrange wavelets generated by the lifting scheme will bemuch like the B-spline Lagrange wavelets. Usually, the Gaussian LagrangeDAF based wavelets are smoother and decay more rapidly than B-splineLagrange wavelets. If we select more sophisticated window shapes, theLagrange wavelets can be generalized further. We shall call theseextensions Bell-windowed Lagrange wavelets.

VISUAL GROUP NORMALIZATION

It is well known that the mathematical theory of wavelet transforms andassociated multiresolution analyses has applications in signalprocessing and engineering problems, where appropriate subband filtersare the central entities. The goal of wavelet signal filtering is topreserve meaningful signal components, while efficiently reducing noisecomponents. To this end, we use known magnitude normalization techniques[28, 29] for the magnitudes of filter coefficients to develop a newperceptual normalization to account for the human vision response.

From a signal processing point of view, wavelet coefficients can beregarded as the results of the signal passing through equivalentdecomposition filters (EDF). The responses of the EDF LC_(j,m)(ω) arethe combination of several recurrent subband filters at differentstages. As shown in FIG. 40, the EDF amplitudes of different sub-blocksare different. Thus the magnitude of the decomposition coefficients ineach of the sub-blocks will not exactly reproduce the true strength ofthe signal components. Stated differently, various EDFs are incompatiblewith each other in the wavelet transform. To adjust the magnitude of theresponse in each block, the decomposition coefficients are re-scaledwith respect to a common magnitude standard. Thus EDF coefficients,C_(j,m)(k), on layer j and block m should be multiplied by a magnitudescaling factor, λ_(j,m), to obtain an adjusted magnitude representation[28]. This factor can be chosen as the reciprocal of the maximummagnitude of the frequency response of the equivalent filter on node(j,m) $\begin{matrix}{{\lambda_{j,m} = \frac{1}{\sup\limits_{\omega \in \Omega}\{ {{{LC}_{j,m}(\omega)}} \}}}\quad,\quad{\Omega = \lbrack {0,{2\quad\pi}} \rbrack}} & (140)\end{matrix}$This idea was recently extended to Group Normalization (GN) of waveletpackets for signal processing [29].

An image can be regarded as the result of a real object processed by ahuman visual system. The latter has essentially many subband filters.The responses of these human filters to various frequency distributionsare not at all uniform. Therefore, an appropriate normalization of thewavelet coefficients is necessary. Actually, the human visual system isadaptive and has variable lens and focuses for different visualenvironments. Using a just-noticeable distortion profile, we canefficiently remove the visual redundancy from decomposition coefficientsand normalize them with respect to a standard of perception importanceas shown in FIGS. 42(a-b). A practical, simple model for perceptionefficiency has been presented by Watson, et al. [5] for datacompression. This model is adapted here to construct the “perceptuallossless” response magnitude Y_(j,m) for normalizing according to thevisual response function [39], $\begin{matrix}{Y_{j,m} = {a\quad 10^{{k{({\log\quad\frac{2^{j}f_{0}d_{m}}{R}})}}^{2}}}} & (141)\end{matrix}$where α defines the minimum threshold, k is a constant, R is the DisplayVisual Resolution (DVR), ƒ₀ is the spatial frequency, and d_(m) is thedirectional response factor. Together with the magnitude normalizedfactor λ_(j,m), this leads to the perceptual lossless quantizationmatrixQ _(j,m)=2Y _(j,m)λ_(j,m)  (142)

This treatment provides a simple, human-vision-based threshold technique[39] for the restoration of the most important perceptual information inan image. For gray-scale image processing, the luminance (magnitude) ofthe image pixels is the principal concern. We refer to the combinationof the above mentioned two normalizations as the Visual GroupNormalization (VGN) of wavelet coefficients.

MASKING TECHNIQUE

Masking is essential to many signal-processing algorithms. Appropriatemasking will result in reduction of noise and undesired components.Certainly, it is very easy to set up masking if the spectraldistribution of a signal and its noise is known. However, in most cases,such prior knowledge is not available. Statistical properties of thesignal and its noise are assumed so that the noise is taken to berelatively more random than the signal in each subband. Hard logicmasking and soft logic masking techniques are discussed in the followingtwo subsections.

Hard Logic Masking

The Visual Group Normalization method provides an efficient approach forre-normalizing the wavelet decomposition coefficients so that varioussubband filters have appropriate perceptual impulse responses. However,this algorithm alone does not yield the best SNR in real signalprocessing. Essentially, various noise and/or interference componentscan be embedded in different nodes of the subband decomposition tree. Toachieve SNR-improved reconstruction of the signal and/or image, afiltering process is needed to reduce the noise and preserve the mainsignal information. Noise due to random processes has a comparativelywide-band distribution over the decomposition tree, whereas mechanicalnoise can have a narrow-band distribution over a few specific subbandcomponents. Therefore, time-varying masking techniques are utilized toreduce noise. We discuss a few useful masking methods in the rest ofthis subsection.

Single Dead-Zone Threshold Masking

A single zone threshold masking is the simplest masking method. With agiven decomposition tree, a constant threshold r is selected for ourmagnitude normalized wavelet decomposition coefficients NC_(j,m)(k).That is, if the absolute value of NC_(j,m)(k) is greater than thethreshold r, the original decomposition coefficient will be kept;otherwise it will be set to zero. That is $\begin{matrix}{{C_{j,m}(k)} = \{ \begin{matrix}{C_{j,m}(k)} & {{{{NC}_{j,m}(k)}} > r} \\0 & {{{{NC}_{j,m}(k)}} < r}\end{matrix} } & (143)\end{matrix}$

This approach is similar to the Pass-Band Selection technique of Ref.[43], used in a FFT framework. However, in the present approach, thedecomposition coefficients are re-scaled using the visual groupnormalization. Therefore, even with a single zone masking, it isexpected that for a given noisy signal the present wavelet analysis willachieve a better SNR than that of a single-band FFT method.

Adaptive Node Mean/Variance Threshold Masking

In practice, we hope the threshold r can be adaptively adjusted to thestrength of the noisy environment. Thus the threshold r should be sethigher to suppress a noisier signal, and in general, r should vary as afunction of the statistical properties of the wavelet decompositioncoefficients, the simplest and most important of which are the mean andsecond variance. These are incorporated in the present work.

We define the mean and second variance of the magnitude of thenormalized coefficients on node (j,m) as $\begin{matrix}{\eta_{j,m} = {\frac{1}{N_{j}}{\sum\limits_{k = 0}^{N_{j} - 1}\quad{{NC}_{j,m}(k)}}}} & (144)\end{matrix}$and $\begin{matrix}{\sigma_{j,m} = ( {\frac{1}{N_{j}}{\sum\limits_{k = 0}^{N_{j} - 1}\quad\lbrack {{{NC}_{j,m}(k)} - \eta_{j,m}} \rbrack^{2}}} )^{\frac{1}{2}}} & (145)\end{matrix}$where N_(j)=2^(−j)N and N is the total length of a filter. A masking isthen set according to the following analysis:

(1) Introduce a factor |α|<1, α∈R.

(2) Set the corresponding wavelet transform coefficients, C_(j,m)(k), tozero, if the inequality, |NC_(j,m)(k)|<|η_(j,m)+ασ_(j,m)| holds. Thisimplies that the magnitude of the normalized coefficients NC_(j,m)(k) isless than the mean, η_(j,m), within the statistical deviation ofασ_(j,m), and hence that NC_(j,m)(k) as a noise component.

(3) Retain the corresponding wavelet transform coefficients, C_(j,m)(k),for reconstruction, satisfying the inequality,|NC_(j,m)(k)|≧|η_(j,m)+ασ_(j,m)|. This implies that the magnitude ofNC_(j,m)(k) is greater than the mean, η_(j,m), within the statisticaldeviation of ασ_(j,m). We consider such a NC_(j,m)(k) as a targetcomponent. The rules (2) and (3) are summarized as $\begin{matrix}{C_{j,{m{(k)}}} = \{ \quad\begin{matrix}{C_{j,m}(k)} & {{{{NC}_{j,m}(k)}} \geq {{\eta_{j,m} + {\alpha\quad\sigma_{j,m}}}}} \\0 & {{{{NC}_{j,m}(k)}} < {{\eta_{j,m} + {\alpha\quad\sigma_{j,m}}}}}\end{matrix}\quad } & (146)\end{matrix}$Adaptive Whole Tree Threshold Masking

For certain applications, it is possible that all NC_(j,m)(k) on aparticular node (j,m) have essentially the same values. Theaforementioned adaptive node mean/variance threshold masking techniquebecomes invalid in such a case. We use an adaptive whole-tree thresholdmasking method for this situation. The basic procedure is very similarto Method 2, except that the mean and second variance are calculated forthe whole tree T, according to $\begin{matrix}{\eta = {\frac{1}{N}{\sum\limits_{j}\quad{\sum\limits_{m}\quad{\sum\limits_{k = 0}^{N_{j} - 1}\quad{{NC}_{j,m}(k)}}}}}} & (147)\end{matrix}$and $\begin{matrix}{\sigma = {{( {\frac{1}{N}{\sum\limits_{j}\quad{\sum\limits_{m}\quad{\sum\limits_{k = 0}^{N_{j} - 1}\quad\lbrack {{{NC}_{j,m}(k)} - \eta} \rbrack^{2}}}}} )^{\frac{1}{2}}\quad( {j,m} )} \in T}} & (148)\end{matrix}$The corresponding reconstruction coefficients are selected by the rules$\begin{matrix}{C_{j,{m{(k)}}} = \{ \begin{matrix}{C_{j,m}(k)} & {{{{NC}_{j,m}(k)}} \geq {{\eta + {\alpha\quad\sigma}}}} \\0 & {{{{NC}_{j,m}(k)}} < {{\eta + {\alpha\quad\sigma}}}}\end{matrix} } & (149)\end{matrix}$Constant False Alarm Masking

In certain applications, such as radar signals generated from a givenenvironment, it is useful to select an alarm threshold, r, based on themean value of multiple measurements of the background signal. Thisapproach is similar to a background-contrasted signal processing inwhich only the differences of the signal's optimal tree decompositioncoefficients, NC_(j,m)(k), and NC_(j,m) ^(β)(k), the backgrounddecomposition coefficients of the same tree structure, are used forbackground-contrasted signal reconstruction.

Softer Logic Masking

The various maskings discussed above can be regarded as hard logicmasking, which are similar to a bias-estimated dead-zone limiter. Jain[16] has shown that a non-linear dead-zone limiter can improve the SNRfor weak signal detectionη(y)=sgn(y)(|y|−δ)₊ ^(β) −1≦β≦1  (150)where δ is a threshold value. Donoho has shown that the β=1 case of theabove expression is a nearly optimal estimator for adaptive NMR datasmoothing and de-noising [9]. Independently, two of the present authors(Shi and Bao) in a previus work [28] have utilized hard logic masking toextract a target from formidable background noise efficiently.

The various threshold cutoffs of multiband expansion coefficients inhard logic masking methods are very similar to the cutoff of a FFTexpansion. Thus, Gibbs oscillations associated with FFTs will also occurin the wavelet transform using a hard logic masking. Although hard logicmasking methods with appropriate threshold values do not seriouslychange the magnitude of a signal after reconstruction, they can causeconsiderable edge distortions in a signal due to the interference ofadditional high frequency components induced by the cutoff. The higherthe threshold value, the larger the Gibbs oscillation will be. Sinceimage edges are especially important in visual perception, hard logicmasking can only be used for weak noise signal (or image) processing,such as electrocardiogram (ECG) signal filtering, where relatively smallthreshold values are required. In this disclosure, we use a Soft LogicMasking (SLM) method. In our SLM approach, a smooth transition band neareach masking threshold is introduced so that any decompositioncoefficients, which are smaller than the threshold value will be reducedgradually to zero, rather than being exactly set to zero. This treatmentefficiently suppresses possible Gibbs oscillations and preserves imageedges, and consequently improves the resolution of the reconstructedimage. The SLM method can be expressed asĈ _(j,m)(k)=sgn(C _(j,m)(k))(|C _(j,m)(k)|−δ)₊ ^(β) ×S({overscore (NC_(j,m) (k))})  (151)where Ĉ_(j,m)(k) are the decomposition coefficients to be retained inthe reconstruction and quantity {overscore (NC_(j,m)(k))} is defined as$\begin{matrix}{\overset{\_}{{NC}_{j,m}(k)} = \frac{{{NC}_{j,m}(k)}}{\max\limits_{{({j,m})} \in T}\{ {{{NC}_{j,m}(k)}} \}}} & (152)\end{matrix}$

The softer logic mapping, S:[0,1]→[0,1], is a non-linear, monotonicallyincreasing sigmoid functional. A comparison of the hard and softer logicmasking functionals is depicted in FIG. 47.

In 2D image processing, it is often important to preserve the imagegradient along the xy-direction. For this purpose, we modify theaforementioned softer logic functional to $\begin{matrix}{{{\hat{C}}_{j,m}(k)} = {{C_{j,m}(k)}{S( \frac{\overset{\_}{{NC}_{j,m}(k)} - \zeta}{1 - \zeta} )}}} & (153)\end{matrix}$where ζ is a normalized adaptive threshold. For an unknown noise level,an useful approximation to ζ is given byζ=γ_(upper) {circumflex over (σ)}√{square root over (2logN/N)}  (154)where {circumflex over (σ)} is a scaling factor conveniently chosen as{circumflex over (σ)}=1/1.349. The quantity γ_(upper) is an upper frameboundary of the wavelet transform, i.e. the upper boundary singularvalue of the wavelet decomposition matrix. Using arguments similar tothose given by Donoho [9], one can show that the above Softer LogicMasking reconstruction is a nearly optimal approximation in the minimaxerror sense.

EXPERIMENTAL RESULTS

To test our new approaches, standard benchmark 512×512 Y-componentimages are employed. Generally, the possible noise sources includephotoelectric exchange, photo spots, the error of image communication,etc. The noise causes the visual perception to generate speckles, blips,ripples, bumps, ringings and aliasing. The noise distortion not onlyaffects the visual quality of the images, but also degrades theefficiency of data compression and coding. De-noising and smoothing areextremely important for image processing.

The traditional image processing techniques can be classified as twokinds: linear or non-linear. The principle methods of linear processingare local averaging, low-pass filtering, band-limit filtering ormulti-frame averaging. Local averaging and low-pass filtering onlypreserve the low band frequency components of the image signal. Theoriginal pixel strength is substituted by an average of its neighboringpixels (within a square window). The mean error may be improved but theaveraging process will blur the silhouette and finer details of theimage. Band-limited filters are utilized to remove the regularlyappearing dot matrix, texture and skew lines. They are useless for noisewhose correlation is weaker. Multi-frame averaging requires that theimages be still, and the noise distribution stationary. These conditionsare violated for motion picture images or for a space (time)-varyingnoisy background.

The traditional image quality is characterized by a mean square error(MSE), which possesses the advantage of a simple mathematical structure.For a discrete signal {s(n)} and its approximation {ŝ(n)}, n=0, . . . ,N, the MSE can be defined to be $\begin{matrix}{{MSE} = {\frac{1}{N}{\sum\limits_{n = 0}^{N - 1}\quad\lbrack {{\hat{s}(n)} - {s(n)}} \rbrack^{2}}}} & (155)\end{matrix}$

However, the MSE based evaluation standard, (such asPSNR=log[(255′255)/MSE]), can not exactly evaluate the image quality ifone neglects the effect of human perception. The minimum MSE rule willcause strong undulations of the image level and destroy the smoothtransition information around the pixels. Commonly used regularizationmethods may degrade the image edges and result in visual blur.

Generally, unsatisfactory traditional image processing is always definedon the whole space (time) region, which does not localize the space(time)-frequency details of the signal. New theoretical research showsthat non-Guassian and non-stationary characteristics are importantcomponents in human visual response. Human visual perception is moresensitive to image edges which consist of sharp-changes of theneighboring gray scale because it is essentially adaptive and hasvariable lens and focuses for different visual environments. To protectedge information as well as remove noise, modern image processingtechniques are predominantly based on non-linear methods. Before thesmoothing process, the image edges, as well as perceptually sensitivetexture must be detected. The commonly used non-linear filteringapproaches include median filtering, and weighting average, etc. Medianfiltering uses the median value within the window instead of theoriginal value of the pixel. This method causes less degradation forslanted functions or square functions, but suppresses the signalimpulses, which are shorter than half of the window length. This willdegrade the image quality. The most serious shortcomings of weightingaverage method are that the weighting-window is not adaptive, andlarge-scale, complicated calculations are required to generate pixelvalues. If the window is made wider, more details will be removed.

More efficient human-vision-system-based image processing techniquespossess the advantages of 1) large range de-correlation for convenienceof compression and filtering; 2) high perceptual sensitivity androbustness; 3) filtering according to human visual response. Ittherefore can enhance the most important visual information, such asedges, while suppressing the large scale of flat regions and background.In addition 4) it can be carried out with real-time processing.

The space (time)-scale logarithmic response characteristic of thewavelet transform is similar to the HVS response. Visual perception issensitive to narrow band low-pass components, and is insensitive to wideband high frequency components. Moreover, from research inneurophysiology and psychophysical studies, the direction-selectivecortex filtering is very much like a 2D-wavelet decomposition. Thehigh-pass coefficients of the wavelet transform can be regarded as thevisible difference predictor (VDP).

Utilizing the modified wavelet analysis-Visual Group-Normalized WaveletTransform (VGN-WT) presented in this paper, we can correct the drawbackthat the raw magnitudes of the transform coefficients do not exactlyyield the perceptual strength of digital images. By use of the softlogic masking technique, the non-linear filtering providesedge-preservation for images, which removes the haziness encounteredwith commonly used filtering techniques.

The first test result is for the so-called “Lena” image, which possessesclear sharp edges, strong contrast and brightness. The second picturetested is “Barbara”. The high texture components and consequently highfrequency edges in the Barbara image create considerable difficultiesfor commonly used filtering techniques. A simple low-pass filter willsmooth out the noise but will also degrade the image resolution, while asimple high-pass filter can enhance the texture edges but will alsocause additional distortion.

We choose 2D half-band Lagrange wavelets as the testing analysis toolsfor image processing. The four 2D wavelets are shown in FIG. 48.

As shown in FIG. 49(a) and FIG. 50(a) respectively, adding Gaussianrandom noise degrades the original Lena and Barbara images. The medianfiltering (with a 3×3 window) result of Lena is shown in FIG. 49(b). Ifis edge-blurred with low visual quality; the speckled noise has beenchanged into bumps. This phenomenon is even more serious for the Barbaraimage, which has many textures and edges. These features always createsevere difficulties for image restoration. As shown in FIG. 49(c) andFIG. 50(c), it is evident that our perceptual DAF wavelet techniqueyields better contrast and edge-preservation results and provides highquality visual restoration performance.

CONCLUSION

This disclosure discusses the design of interpolating wavelets based onLagrange functions, as well as their application in image processing. Aclass of biorthogonal Lagrange interpolating wavelets is studied withregard to its application in signal processing (especially for digitalimages). The most attractive property of the interpolating wavelets isthat the wavelet multiresolution analysis is realized by discretesampling. Thus pre- and post-conditioning processings are not needed foran accurate wavelet analysis. The wavelet coefficients are obtained fromlinear combinations of sample values rather than from integrals, whichimplies the possibility of using parallel computation techniques.

Theoretically, our approach is closely related to the finite elementtechnique for the numerical solution of partial differential equations,the subdivision scheme for interpolation approximations, multi-gridmethods and surface fitting techniques. In this paper, we generalize thedefinition of interpolating Lagrange wavelets and produce threedifferent biorthogonal interpolating Lagrange wavelets, namely HalfbandLagrange wavelets, B-spline Lagrange wavelets and Gaussian-Lagrange DAFwavelets.

Halfband Lagrange wavelets can be regarded as an extension of Dubucinterpolating functionals, auto-correlation shell wavelet analysis andhalfband filters. B-spline Lagrange Wavelets are generated by B-splinewindowing of a Lagrange functional, and lead to increased smoothness andlocalization compared to the basic Lagrange wavelets.

Lagrange Distributed Approximating Functionals (LDAF) can be regarded asscaling functions (wavelet-DAFs) and associated DAF-wavelets can begenerated in a number of ways [41]. Both DAFs and DAF-wavelets aresmoothly decay in both time and frequency representations. The presentwork extends the DAF approach to signal and image processing byconstructing new biorthogonal DAF-wavelets and associated DAF-filtersusing a lifting scheme [32].

In the first part of our image processing application, we combine twoimportant techniques, the coefficient normalization method and softerlogic visual masking based on Human Vision Systems (HVS). The resultingcombined technique is called Visual Group Normalization (VGN) processing[31]. The concept of Visual Loss-less Quantization (VLQ) presented in[39] can lead to a potential breakthrough compared to the traditionalShannon rate-distortion theory in information processing.

We also employ a modified version of Donoho's Soft Threshold method forimage restoration, termed the Softer Logic Perceptual Masking (SLM)technique, for dealing with extremely noisy backgrounds. This techniquebetter preserves the important visual edges and contrast transitionportions of an image than the traditional Donoho method and is readilyadaptable to human vision.

Computational results show that our Lagrange wavelet based VGNprocessing is extremely efficient and robust for digital image blindrestoration and yields the best performance of which we are aware, whenapplied to standard Lena and Barbara images.

REFERENCES

-   [1] R. Ansari, C. Guillemot, and J. F. Kaiser, “Wavelet construction    using Lagrange halfband filters,” IEEE Trans. CAS, vol. 38, no. 9,    pp. 1116-1118, 1991.-   [2] R. Baraniuk, D. Jones, “Signal-dependent time-frequency analysis    using a radially Gaussian kernel,” Signal Processing, Vol. 32, pp.    263-284, 1993.-   [3] C. M. Brislawn, “Preservation of subband symmetry in multirate    signal coding,” IEEE Trans. SP, vol. 43, no. 12, pp. 3046-3050,    1995.-   [4] C. K. Chui, An Introduction to Wavelets, Academic Press, New    York, 1992.-   [5] C. K. Chui, Wavelets: A Tutorial in Wavelet Theory and    Applications, Academic Press, New York, 1992.-   [6] I. Daubechies, “Orthonormal bases of compactly supported    wavelets”, Comm. Pure and Appl. Math., vol. 41, no. 11, pp. 909˜996,    1988.-   [7] I. Daubechies, “The wavelet transform, time-frequency    localization and signal analysis,” IEEE Trans. Inform. Theory, Vol.    36, No. 5, pp. 961-1003, September 1990.-   [8] G. Deslauriers, S. Dubuc, “Symmetric iterative interpolation    processes,” Constructive Approximations, vol. 5, pp. 49-68, 1989.-   [9] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.    Information Theory, vol. 41, no. 3, pp. 613˜627, 1995.-   [10] D. L. Donoho, “Interpolating wavelet transform,” Preprint,    Stanford Univ., 1992.-   [11] S. Dubuc, “Interpolation through an iterative scheme”, J. Math.    Anal. and Appl., vol. 114, pp. 185˜204, 1986.-   [12] A. Harten, “Multiresolution representation of data: a general    framework,” SIAM J. Numer. Anal., vol. 33, no. 3, pp. 1205-1256,    1996.-   [13] C. Herley, M. Vetterli, “Orthogonal time-varying filter banks    and wavelet packets,” IEEE Trans. SP, Vol. 42, No. 10, pp.    2650-2663, October 1994.-   [14] C. Herley, Z. Xiong, K. Ranchandran and M. T. Orchard, “Joint    Space-frequency Segmentation Using Balanced Wavelet Packets Trees    for Least-cost Image Representation,” IEEE Trans. Image Processing,    vol. 6, pp. 1213-1230, September 1997.-   [15] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri,    “Analytic banded approximation for the discretized free    propagator,” J. Physical Chemistry, vol. 95, no. 21, pp. 8299-8305,    1991.-   [16] L. C. Jain, N. M. Blachman, and P. M. Chapell, “Interference    suppression by biased nonlinearities,” IEEE Trans. IT, vol. 41, no.    2, pp. 496-507, 1995.-   [17] N. Jayant, J. Johnston, and R. Safranek, “Signal compression    based on models of human perception”, Proc. IEEE, vol. 81, no. 10,    pp. 1385˜1422, 1993.-   [18] J. Kovacevic, and M. Vetterli, “Perfect reconstruction filter    banks with rational sampling factors,” IEEE Trans. SP, Vol. 41, No.    6, pp. 2047-2066, June 1993.-   [19] J. Kovacevic, W. Swelden, “Wavelet families of increasing order    in arbitrary dimensions,” Submitted to IEEE Trans. Image Processing,    1997.-   [20] A. F. Laine, S. Schuler, J. Fan and W. Huda, “Mammographic    feature enhancement by multiscale analysis,” IEEE Trans. MI, vol.    13, pp. 725-740, 1994.-   [21] S. Mallat, “A theory for multiresolution signal decomposition:    the wavelet representation,” IEEE Trans. PAMI, Vol. 11, No. 7, pp.    674-693, July 1989.-   [22] Y. Meyer, Wavelets Algorithms and Applications, SIAM Publ.,    Philadelphia 1993.-   [23] K. Ramchandran, M. Vetterli, “Best wavelet packet bases in a    rate-distortion sense,” IEEE Trans. Image Processing, Vol. 2, No. 2,    pp. 160-175, April 1993.-   [24] K. Ramchandran, Z. Xiong, K. Asai and M. Vetterli, “Adaptive    Transforms for Image Coding Using Spatially-varying Wavelet    Packets,” IEEE Trans. Image Processing, vol. 5, pp. 1197-1204, July    1996.-   [25] O. Rioul, M. Vetterli, “Wavelet and signal processing,” IEEE    Signal Processing Mag., pp. 14-38, October 1991.-   [26] N. Saito, G. Beylkin, “Multiscale representations using the    auto-correlation functions of compactly supported wavelets,” IEEE    Trans. Signal Processing, Vol. 41, no. 12, pp. 3584-3590, 1993.-   [27] M. J. Shensa, “The discrete wavelet transform: wedding the a    trous and Mallat algorithms”, IEEE Trans. SP, vol. 40, no. 10, pp.    2464˜2482, 1992.-   [28] Z. Shi, Z. Bao, “Group-normalized processing of complex wavelet    packets,” Science in China (Serial E), Vol. 26, No. 12, 1996.-   [29] Z. Shi, Z. Bao, “Group-normalized wavelet packet signal    processing”, Wavelet Application IV, SPIE, vol. 3078, pp. 226˜239,    1997.-   [30] Z. Shi, Z. Bao, “Fast image coding of interval interpolating    wavelets,” Wavelet Application IV, SPIE, vol. 3078, pp. 240-253,    1997.-   [31] Z. Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Perceptual    image processing using Gauss-Lagrange distributed approximating    functional wavelets,” submitted to IEEE SP Letter, 1998.-   [32] W. Swelden, “The lifting scheme: a custom-design construction    of biorthogonal wavelets,” Appl. And Comput. Harmonic Anal., vol. 3,    no. 2, pp. 186˜200, 1996.-   [33] T. D. Tran, R. Safranek, “A locally adaptive perceptual masking    threshold model for image coding,” Proc. ICASSP, 1996.-   [34] M. Unser, A. Adroubi, and M. Eden, “The L₂ polynomial spline    pyramid,” IEEE Trans. PAMI, vol. 15, no. 4, pp. 364-379, 1993.-   [35] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part I: system-theoretic fundamentals,” IEEE    Trans. SP, Vol. 43, No. 5, pp. 1090-1102, May 1995.-   [36] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part II: the FIR case, factorizations, and    biorthogonal lapped transforms,” IEEE Trans. SP, Vol. 43, No. 5, pp.    1103-1115, May 1995.-   [37] M. Vetterli, C. Herley, “Wavelet and filter banks: theory and    design,” IEEE Trans. SP, Vol. 40, No. 9, pp. 2207-2232, September    1992.-   [38] J. D. Villasenor, B. Belzer, and J. Liao, “Wavelet filter    evaluation for image processing,” IEEE Trans. IP, vol. 4, no. 8, pp    1053-1060, 1995.-   [39] A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor,    “Visibility of wavelet quantization noise,” IEEE Trans. Image    Processing, vol. 6, pp. 1164-1175, 1997.-   [40] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,    “Lagrange distributed approximating Functionals,” Physical Review    Letters, Vol. 79, No. 5, pp. 775˜779, 1997.-   [41] G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Wavelets and    distributed approximating functionals,” submitted to Phys. Rev.    Lett.-   [42] Z. Xiong, K. Ramchandran and M. T. Orchard, “Space-frequency    Quantization for Wavelet Image Coding,” IEEE Trans. Image    Processing, vol. 6, pp. 677-693, May 1997.-   [43] S. H. Zhang, Z. Bao, etc. “Target extraction from strong    clutter background,” Tech. Rep., National Key Lab. of Radar Signal    Processing, Xidian University, 1994

IMAGE ENHANCEMENT NORMALIZATION Introduction

Medical image, such computed tomography (CT), magnetic resonance image(MRI), X-ray mammogram, ultrasound and angiography, is one of majormethods for field diagnosis. In particular, X-ray mammogram is widelyrecognized as being the only effective method for the early detection ofbreast cancer. Major advances in screen/film mammograms have been madeover the past decade, which result in significant improvements in imageresolution and film contrast without much increase in X-ray dose. Infact, mammogram films have the highest resolution in comparison tovarious other screen/film techniques. However, many breast cancerscannot be detected just based on mammogram images because of poorvisualization quality of the image. This is due to the minor differencein X-ray attenuation between normal glandular tissues and malignantdisease, which leads to the low-contrast feature of the image. As aresult, early detection of small tumors, particularly for younger womenwho have denser breast tissue, is still extremely difficult.

Mammogram image processing has drawn a great deal of attention in thepast decades [1-9]. Most work focuses on noise reduction and featureenhancement. The statistical noise level is relatively low for imagesobtained by the modern data acquisition techniques. Therefore featureenhancement is more essential for mammogram quality improvement. Sincean image noise reduction algorithm was reported earlier [10], we shallfocus on image enhancement in this work. In our opinion, there are twobasic ideas for mammogram feature improvement. One type of methods is tochange image spectrum distribution so as to increase image edgedensities. As a result, the image appears sharper. The other approach isthe so-called image windowing, in which linear or nonlinear grayscalemappings are defined so as to increase/compress image gradient densityaccording to device.

In an early work, one of present authors realized that the frequencyresponses of wavelet transform subband filters are not uniform. Amagnitude normalization algorithm was proposed to account for thisuniform feature [11, 12]. This idea was later extended to groupnormalization for the wavelet packet transform [13] and visual groupnormalization for still image noise reduction [10] by present authors.In this work we further extend this idea to achieve image enhancement.

Our first approach is based on a rearrangement of image spectrumdistribution by edge enhancement normalization of wavelet coefficients.Our second technique utilized a multiscale functional to obtained deviceadapted visual group normalization of wavelet coefficients. Numericalexperiments indicate that our normalization approach provides excellentenhancement for low quality mammogram image in combination with the useof the biorthogonal interpolating wavelets [10] generated by GaussianLagrange distributed approximating functionals [14].

EDGE ENHANCEMENT NORMALIZATION

Mallat and Zhong realized that Wavelet multiresolution analysis providesa natural characterization for multiscale image edges, and thosemanipulations can be easily achieved by various differentiations [15].Their idea was extended by Laine et al [7] to develop directional edgeparameters based on subspace energy measurement. An enhancement schemebased on complex Daubechies wavelets was proposed by Gagnon et al. [9].These authors made use of the difference between real part and imaginarypart of the wavelet coefficients. One way or another, distorted wavelettransforms are designed to achieve desired edge enhancement.

Our starting point is magnitude normalized or visual group normalizedwavelet subband coefficients NC_(j,m)(k) [10, 12]. We define anenhancement functional E_(j,m)E _(j,m)=α_(j,m)+β_(j,m)Δ  (156)where Δ is the Laplacian and −1≦α_(j,m), β_(j,m)≦1. Coefficientsα_(j,m), β_(j,m) can be easily chosen so that desired image features areemphasized. In particular it enables us to emphasize an image edge ofselected grain size. We note that a slightly modification of α_(j,m) andβ_(j,m) can result in orientation selected image enhancement. A detaileddiscussion of this matter will be presented elsewhere. An overallre-normalization is conducted after image reconstruction to preserve theenergy of the original image. We call this procedure enhancementnormalization.

DEVICE ADAPTED VISUAL GROUP NORMALIZATION

Contrast stretching is old but quite efficient method for featureselective image display. Nonlinear stretching has been used by manyauthors [3, 7, and 16]. Lu and coworkers [16] has recently designedhyperbolic function g_(j)(k)=[tenh(ak−b)+tanh(b)]/[tanh(a−b)+tanh(b)]for wavelet multiscale gradient transformation. Their method works wellfor moon images. The basic idea is to use gradient operators to shape afat image data so that desired portion of the image is projected into ascreen window. However, in most approaches, the perceptual response ofhuman visual system is not appropriately accounted. The human visualsystem is adaptive and has variable lens and focuses for differentvisual environments. We propose a human visual response correctedgrayscale gradient mapping technique for selected contrast enhancement.

The human visual system is adaptive and has variable lens and focusesfor different visual environments. Using a just-noticeable distortionprofile, one can efficiently remove the visual redundancy from thedecomposition coefficients [17] and normalize them with respect to theimportance of perception. A practical simple model for perceptionefficiency has been presented to construct the “perceptual lossless”response magnitude Y_(j,m) for normalizing according to visual response,$\begin{matrix}{Y_{j,m} = {a10}^{{k{({\log\quad\frac{2^{j}f_{n}d_{m}}{R}})}}^{2}}} & (157)\end{matrix}$where k is a constant, R is the Display Visual Resolution (DVR), ƒ₀ isthe spatial frequency, and d_(m) is the directional response factor. Aperceptual lossless quantization matrix Q_(j,m) is [10]Q _(j,m)=2Y _(j,mj,m)   (158)where λ_(j,m) is a magnitude normalized factor. This treatment providesa simple human-vision-based threshold technique for the restoration ofthe most important perceptual information in an image. For grayscaleimage contrast stretching, we first appropriately normalize thedecomposition coefficients according to the length scale, L, of thedisplay device [16] so that they fall in interval of [0,1] of the deviceframeNC _(j,m)(k)=Q _(j,m) C _(j,m)(k)/L   (159)

We then use a nonlinear mapping to obtain desired contrast stretching{overscore (NC_(j,m))}=γ_(j,m) X _(j,m)(NC _(j,m))   (160)where constant γ_(j,m) and function X_(j,m) is appropriately chosen sothat desired portion of the grayscale gradient is stretched orcompressed. For example, function $\begin{matrix}{{X_{j,m}(x)} = \frac{{\tan\quad{a_{j,m}( {x - b_{j,m}} )}} + {\tan\quad a_{j,m}b_{j,m}}}{{\tan\quad{a_{j,m}( {1 - b_{j,m}} )}} + {\tan\quad\alpha_{j,m}b_{j,m}}}} & (161)\end{matrix}$can been tuned to stretch any portion the of grayscale gradient. Thisprocedure shall be called device adapted visual group normalization.

EXPERIMENTAL RESULT

To test our new approaches, low-contrast and low quality breastmammogram images are employed. Mammograms are complex in appearance andsigns of early disease are often small or subtle. Digital mammogramimage enhancement is particularly important for aiding radiologists andfor the development of automatically detecting expert system. A typicallow quality front-view image is shown in FIG. 51(a). The original imageis coded at 512×512 pixel size with 2 bytes/pixel and 12 bits of grayscale. We have conducted edge enhancement normalization and deviceadapted visual group normalization. As shown in FIG. 51(b), there is asignificant improvement in both edge representation and image contrast.In particular, the domain and internal structure of high-density cancertissues are clear displayed. FIG. 52(a) is an original 1024×1024side-view breast image which has been digitized to 200 micron pixel edgewith 8 bits of gray scale. Enhanced image result is depicted in FIG.52(b). In this case we obtain a similar result in the previous one.

CONCLUSION

Edge enhancement normalization (EEN) and device adapted visual groupnormalization (DAVGN) are proposed for image enhancement without priorknowledge of the spatial distribution of the image. Our algorithm is anatural extension of earlier normalization techniques for imageprocessing. Biorthogonal interpolating distributed approximatingfunctional wavelets are used for our data representation. Excellentexperimental performance is found for digital mammogram imageenhancement.

REFERENCES

-   [1] R. Gordon and R. M. Rangayan, “Feature enhancement of film    mammograms using fixed and adaptive neighborhoods,” Applied Optics,    vol. 23, no. 4, pp. 560-564, February 1984.-   [2] A. P. Dhawan and E. Le Royer, “Mammographic feature enhancement    by computerized image processing,” Comput. Methods and Programs in    Biomed., vol. 27, no. 1, pp. 23-35, 1988.-   [3] P. G. Tahoces, J. Correa, M. Souto, C. Gonzalez, L. Gomez,    and J. J. Vidal, “Enhancement of chest and breast radiographs by    automatic spatial filtering,” IEEE Trans. Med. Imag., vol. 10, no.    3, pp. 330-335, 1991.-   [4] S. Lai, X. Li, and W. F. Bischof, “On techniques for detecting    circumscribed masses in mammograms,” IEEE Trans. Med. Imag., vol. 8,    no. 4, pp. 377-386, 1989.-   [5] M. Nagao and T. Matsuyama, “Edge preserving smoothing,” Comput.    Graphics and Image Processing, vol. 9, no. 4, pp. 394-407, 1979.-   [6] A. Scheer, F. R. D. Velasco, and A. Rosenfield, “Some new image    smoothing techniques,” IEEE Trans. Syst., Man. Cyber., vol. SMC-10,    no. 3, pp. 153-158, 1980.-   [7] A. F. Laine, S. Schuler, J. Fan, and W. Huda, “Mammographic    feature enhancement by multiscale analysis,” IEEE Trans. Med. Imag.,    vol. 13, no. 4, 1994.-   [8] J. Lu, D. M. Healy Jr., and J. B. Weaver, “Contrast enhancement    of medical images using multiscale edge representation,” Optical    Engineering, in press.-   [9] L. Gagnon, J. M. Lina, and B. Goulard, “Sharpening enhancement    of digitized mammograms with complex symmetric Daubechies wavelets,”    preprint.-   [10] Zhuoer Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman,    “Percepual image processing using Gaussian-Lagrange distributed    approximating functional wavelets,” IEEE SP Lett., submitted.-   [11] Zhuoer Shi and Zheng Bao, “Group-normalized processing of    complex wavelet packets,” Science in China, Ser. E, Vol. 40, No. 1,    pp. 28˜43, February, 1997.-   [12] Zhuoer Shi and Zheng Bao, “Group-normalized wavelet packet    signal processing,” SPIE, Vol. 3078, pp. 226˜239, 1997-   [13] Zhuoer Shi and Zheng Bao, “Group normalized wavelet packet    transform,” IEEE Trans. CAS II, in press, 1998.-   [14] G. W. Wei, D. S. Zhang, D. J. Kouri and D. K. Hoffman,    “Lagrange distributed approximating functionals,” Phys. Rev. Lett.,    vol. 79, no. 5, pp. 775-779, 1997.-   [15] S. Mallat, and S. Zhong, “Characterization of Signals from    multiscale edges,” IEEE Trans. PAMI, vol. 14, no. 7, pp. 710-732,    1992.-   [16] J. Lu, and D. M. Healy, Jr., “Contrast enhancement via    multiscale gradient transform,” preprint.-   [17] A. B. Watson, G. Y. Yang, J. A. Solomon and J. Villasenor,    “Visibility of wavelet quantization noise,” IEEE Trans. Image    Processing, vol. 6, pp. 1164-1175, 1997.

VARYING WEIGHT TRIMMED MEAN FILTER FOR THE RESTORATION OF IMPULSECORRUPTED IMAGES Introduction

Images are often corrupted by impulse noise due to a noisy sensor orchannel transmission errors. Impulse interference may be broadly definedas corruption which is random, sparse, and of high or low amplituderelative to local signal values. Impulse noise seriously affects theperformance of various signal processing techniques, e.g., edgedetection, data compression, and pattern recognition. One of the tasksof image processing is to restore a high quality image from a corruptedone for use in subsequent processing. The goal of image filtering is tosuppress the noise while preserving the integrity of significant visualinformation such as textures, edges and details. Nonlinear techniqueshave been found to provide more satisfactory results than linearmethods. One of the most popular nonlinear filters is the median filter[1-2], which is well-known to have the required properties ofsuppressing the impulse noise and preserving the edges. However, it isalso true that the median filter is not optimal. It suppresses the truesignal as well as noise in many applications. Many modifications andgeneralizations of the median filter have been proposed [3-8] forimproving its performance.

In reference [8], we presented a generalized trimmed mean filter (GTMF),which is essentially a generalization to the alpha-trimmed mean filter(α-TMF) [5-6]. In GTMF, a “median basket” is employed to select apredetermined number of pixels on both sides of the median value to thesorted pixels of the moving window. The luminance values of the selectedpixels and the value of the center pixel in the window are then weightedand averaged to give the filtering output. As mentioned in [8], it isimportant to have the center pixel participate in the averagingoperation when removing additive noise, but one usually sets the weightof the center pixel to zero when filtering impulse-noise-corruptedimages.

Although the GTMF outperforms many well known methods in removing highlyimpulse noise corrupted images, it can be further modified to improvethe filtering performance. In GTMF, the averaging weights arepredetermined and fixed throughout the filtering procedure. In thispaper, a varying weight function is designed and applied to GTMF. Wegive it a new name, varying weight trimmed mean filter (VWTMF). InVWTMF, the argument of the weight function is the absolute differencebetween the luminance values in the median basket and the median value.Because we only concentrate on filtering impulse-noise-corrupted imagesin this paper, the weight of the center pixel in the moving window isalways assumed to be zero. For most effectively removing impulse noise,we combine the VWTMF with a switching scheme [9] as a impulse detectorand an iterative procedure [8] is employed to improve the filteringperformance.

Varying Weight Trimmed Mean Filter

The pixels {I₁, I₂, . . . , I_(m−1), I_(m), I_(m+1), . . . , I_(n)} inthe moving window associated with a pixel I_(c), have been sorted in anascending (or descending) order in the same way as in the conventionalmedian filtering technique, with I_(m) being the median value. The keygeneralization to the median filter which is introduced in thealpha-trimmed mean filter (α-TMF) [5,6] is to design a median basket inwhich to collect a given number of pixels above and below the medianpixel. The values of these pixels are then averaged to give thefiltering output, A_(c), as an adjusted replacementvalue to I_(c),according to $\begin{matrix}{A_{c} = {\frac{1}{{2\quad L} + 1}{\sum\limits_{j = {m - L}}^{m + L}I_{j}}}} & (162)\end{matrix}$where L=└αn┘, with 0≦α≦0.5. It is evident that a single-entry medianbasket (L=0) α-TMF is equivalent to the median filter and a n-entrymedian basket (L=≦└0.5┘) is equivalent to the simple moving averagefilter. The generalized trimmed mean filter (GTMF) [8] uses a medianbasket in the same way as is does in the α-TMF to collect the pixels.The luminance values in the median basket and the center pixel I_(c) inthe window are thenweighted and averaged to give the GTMF output:$\begin{matrix}{G_{c} = \frac{{w_{c}I_{c}} + {\sum\limits_{j = {m - L}}^{m + L}{w_{j}I_{j}}}}{w_{c} + {\sum\limits_{j = {m - L}}^{m + L}w_{j}}}} & (163)\end{matrix}$where G_(c) is the GTMF output, and w_(c) and the w_(j)'s are theweights. It is interesting to see that when w_(c)=0 and all w_(j)'s areequal to each other (nonzero), the GTMF becomes the α-TMF. When allweights except w_(m) are equal to zero, it becomes the standard medianfilter. In the GTMF, the values of the weights are predetermined beforefiltering and are fixed during the filtering procedure. However, it ispossible to further improve the GTMF by varying the weights according toabsolute difference between the luminance values in the median basketand the median value. For the removal of impulse noise, we set w_(c)=0and modify the GTMF by varying the weights, so the varying weighttrimmed mean filter (VWTMF) is given by $\begin{matrix}{V_{c} = \frac{\sum\limits_{j = {m - L}}^{m + L}{{w( x_{j} )}I_{j}}}{\sum\limits_{j = {m - L}}^{m + L}{w( x_{j} )}}} & (164)\end{matrix}$where, x_(j) is a value in the range of [0,1] defined by $\begin{matrix}{x_{j} = \frac{{I_{j} - I_{m}}}{B}} & (165)\end{matrix}$with B being the maximum pixel value of a given type of image (e.g.,B=255 for a 8 bit gray-scale image). The weight w(x) in Equation (165)is a decreasing function in the range [0,1] and is taken to be$\begin{matrix}{{w(x)} = {\mathbb{e}}^{- {A{(\frac{x}{x - 1})}}^{2}}} & (166)\end{matrix}$

The above weight function for A=2 is displayed in FIG. 53. Notice thatw(0)=1 and w(1)=0, w(x_(m))=1 is always the largest weight and thegreater the difference between the pixel value and the median value, thesmaller will be the weight. We anticipate that the VWTMF will outperformboth the median filter and the α-TMF in suppressing impulse noise, whilepreserving the image edges. As is well known, the median value has theleast probability to be impulse noise corrupted because the impulses aretypically presented near the two ends of the sorted pixels. However, themedian value may not be optimal because it may differ significantly fromthe noise-free value. The α-TMF will not perform better than the medianfilter when treating impulse noise corrupted images in the absence ofany other technique (such as the switching scheme) because the corruptedpixels may be also selected to the median basket for the averagingoperation. In contrast, the VWTMF which uses a weighted averagingoperation can alleviate the shortcomings of both filters. The weight ofthe median value is the large stand the weights of other luminancevalues in the median basket vary according to their differences from themedian value. If a impulse corrupted pixel I_(j) happens to be selectedfor inclusion in the median basket, its contribution to the average willbe small because x_(j) is large. In general, the weight function canassist the VWTMF in eliminating the impulse noise while providing a welladjusted replacement value for the center pixel I_(c).

It must be noted that although the present implementation of the VWTMFemploys Equation (166) as the weight function, other monotonicallydecaying functions in the range of [0,1] may also be selected.

THE SWITCHING SCHEME BASED ITERATIVE METHOD

Many algorithms have been proposed to detect and replace impulse noisecorrupted pixels of a image [9-12]. In this paper, a switching schemesimilar to that used in reference [9], based on the VWTMF, is employedto detect impulse noise and recover the noise-free pixels. The filteringoutput I_(c), is generated according to the following algorithm$\begin{matrix}{I_{c}^{\prime} = \{ \begin{matrix}{I_{c}^{(i)},} & {{{I_{c}^{(i)} - V_{c}}} < T} \\{V_{c},} & {{{I_{c}^{(i)} - V_{c}}} \geq T}\end{matrix} } & (167)\end{matrix}$where I_(c) ^((i)) is the initial input value and V_(c) is given inEquation (164). A threshold T is chosen to characterize the absolutedifference between I_(c) ^((i)) and V_(c). If the difference is largerthe threshold, it implies that the pixel differs significantly from itsneighbors. It is therefore identified as an impulse noise corruptedpixel, and is then replaced by V_(c). If the difference is smaller thanthe threshold, it implies that the original is similar to itsstatistical neighbors, and we identify it as noise-free pixel, and ittherefore retains its original value.

Iteration of the above scheme will further improve the filteringperformance, especially for images that are highly corrupted by impulsenoise. The iteration procedure can be expressed as $\begin{matrix}{{I_{c}(t)} = \{ \begin{matrix}{I_{c}^{(i)},} & {{{I_{c}^{(i)} - {V_{c}(t)}}} < T} \\{{V_{c}(t)},} & {{{I_{c}^{(i)} - {V_{c}(t)}}} \leq T}\end{matrix} } & (168)\end{matrix}$where I_(c)(t) is the system output at time t with I_(c)(0)=I_(c)^((i)), V_(c)(t) is the VWTMF filtering output given by $\begin{matrix}{{V_{c}(t)} = \frac{\sum\limits_{j = {m - L}}^{m + L}{{w( x_{j} \middle| {t - 1} )}{I_{j}( {t - 1} )}}}{\sum\limits_{j = {m - L}}^{m + L}{w( x_{j} \middle| {t - 1} )}}} & (169)\end{matrix}$

Note that it is important for the iterative procedure always to compareV_(c)(t) with the initial input I_(c) ^((i)) and to update the outputwith I_(c) ^((i)) when their absolute difference is less than thethreshold T.

NUMERICAL EXPERIMENTS

The standard 8 bit, gray-scale “Lena” image (size 512×512) is used as anexample to test the usefulness of our filtering algorithm. We degradedit with various percentages of fixed-value (0 or 255) impulse noise. Theproposed algorithm is compared with the median filtering and α-TMFalgorithms, and their peak signal-to-noise ratio (PSNR)performances arelisted in TABLE 3.

TABLE 3 Comparative Filtering Results in PSNR for Lena Image Corruptedwith Different Amount of Fixed Impulse Noise Algorithm* 15% 20% 25% 30%35% 40% Median 32.19 31.48 30.69 29.91 29.37 28.75 dB dB dB dB dB dBα-TMF 32.09 31.30 30.39 29.28 28.38 27.49 dB dB dB dB dB dB VWTMF 32.3131.67 30.89 30.12 29.63 29.06 dB dB dB dB dB dB Median Switch 35.2033.87 32.91 31.84 30.99 30.29 dB dB dB dB dB dB α-TMF Switch 36.04 34.9333.97 32.59 31.67 30.72 dB dB dB dB dB dB VWTMF Switch 36.34 35.13 34.2933.15 32.24 31.43 dB dB dB dB dB dB *All algorithms are implementedrecursively for optimal performance.Both direct and switch-based application of the filters are presented inTABLE 3 for comparison, and all algorithms are implemented recursivelyin a 3×3 window for optimal performance (The same iterative procedure isused for all the switching scheme based algorithms.) A 3-entry medianbasket (L=1) is used for both α-TMF and VWTMF algorithms. The A used inthe VWTMF weight function is 2 for images with 15%, 20%, 25%, and 30%impulse noise, and is 3 with 35% impulse noise, and is 4.5 with 40%impulse noise. The threshold used for the switching schemes is 28.

From TABLE I, it is easy to see that without the switching scheme, theα-TMF performs even worse than the median filter. However, it performsbetter than the median filter when the switching scheme is used. Thisreflects the fact that although the α-TMF may not perform well inimpulse noise removal, it is a good impulse detector. The VWTMF performsbetter than either the median filter or the α-TMF, whether the switchingscheme is used or not. It is simple, robust and efficient. The VWTMFperforms well in removing impulse noise, and is simultaneously a goodimpulse detector. It is especially efficient for filtering highlyimpulse noise corrupted images. FIG. 54 shows the original noise-freeimage, the impulse noise corrupted image (40% impulse noise), and thefiltered results for several algorithms. One can observe from FIG. 54that even if there is no switching scheme employed, the VWTMF performsbetter than either the median filter or the α-TMF in terms ofsuppressing noise and preserving edges. The α-TMF performs even worsethan the median filter when no switching scheme is employed. Manyspeckles still remain in the α-TMF filtered image. However, theVWTMF-based switching scheme provides a result that is almost the sameas the original noise-free image.

CONCLUSIONS

This disclosure presents a new filtering algorithm for removing impulsenoise from corrupted images. It is based on varying the weights of thegeneralized trimmed mean filter (GTMF)continuously according to theabsolute difference between the luminance values of selected pixels inthe median basket and the median value. Numerical results show that theVWTMF is robust and efficient for noise removal and as an impulsedetector. Although the VWTMF is only used for removing impulse noise inthis disclosure, we expect that it also will be useful for removingadditive noise.

REFERENCES

-   [1] A. Rosenfeld and A. C. Kak, Digital Picture Processing, New    York: Academic Press, 1982, Vol. 1.-   [2] I. Pitas, Digital Image Processing Algorithms, Prentice Hall,    1993.-   [3] D. R. K. Brownring, “The weighted median filter,” Comm. Assoc.    Comput. Mach., Vol. 27, pp. 807-818, 1984.-   [4] H. M. Lin and A. N. Willson, “Median filters with adaptive    length,” IEEE Trans.

Circuits Syst., Vol. 35, pp. 675-690, 1988.

-   [5] J. B. Bednar and T. L. Watt, “Alpha-trimmed means and their    relationship to median filters,” IEEE Trans. Acoust., Speech, and    Signal Processing, Vol. ASSP-32, pp. 145-153, 1984.-   [6] Y. H. Lee, S. A. Kassam, “Generalized median filtering and    related nonlinear filtering techniques,” IEEE Trans. Acoust.,    Speech, and Signal Processing, Vol. ASSP-33, pp. 672-683, 1985.-   [7] P. K. Sinha and Q. H. Hong, “An improved median filter”, IEEE    Trans. Medical Imaging, Vol. 9, pp. 345-346, 1990.-   [8] D. S. Zhang, Z. Shi, D. J. Kouri, and D. K. Hoffman, “A new    nonlinear image filtering technique,” Optical Engr., Submitted.-   [9] T. Sun and Y. Neuvo, “Detail-preserving median based filters in    image processing,” Pattern Recognition Letter, Vol. 15, pp. 341-347,    1994.-   [10] R. Sucher, “Removal of impulse noise by selective filtering,”    IEEE Proc. Int. Conf. Image Processing, Austin, Tex., November 1994,    pp. 502-506.-   [11] E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, “A new    efficient approach for the removal of impulse noise from highly    corrupted images,” IEEE Trans. Image Processing, Vol. 5, pp.    1012-1025, 1996.-   [12] Z. Wang and D. Zhang, “Restoration of impulse noise corrupted    images using long range correlation,” IEEE Signal Processing Letter,    Vol. 5, pp. 4-7, 1998.

A NEW NONLINEAR IMAGE FILTERING TECHNIQUE Introduction

Images are often corrupted by noise that seriously affects theperformance of various signal processing techniques, data compression,and pattern recognition. The goal of noise filtering is to suppress thenoise while preserving the integrity of significant visual informationsuch as textures, edges and details. Linear local averaging filters areessentially low pass filters. Because the impulse responses of the lowpass filters are spatially invariant, rapidly changing signals such asimage edges and details, cannot be well preserved. Consequently, impulsenoise cannot be effectively removed by linear methods; nonlineartechniques have been found to provide more satisfactory results. Some ofthe most popular nonlinear filters are the median filter [1], and itsvarious generalizations [2-8], which are well known to have the requiredproperties for edge preservation and impulse noise removal. However, themedian filter is not optimal since it is typically implemented uniformlyacross the image. It suppresses the true signal as well as noise in manyapplications. In the presence of impulse noise, the median filter tendsto modify pixels that are not degraded. Furthermore, it is prone toproduce edge jitter when the percentage of impulse noise is large. Inorder to improve the performance of the median filter, two median basedfilters, namely the α-trimmed mean (α-TM) [5-6] filter and the modifiedtrimmed mean (MTM) [6] filter, which select from the window only theluminance values close to the median value, have been proposed. Theselected pixels are then averaged to provide the filtering output.Although superior to the median filter in some applications, thesealgorithms still have problems. In general, the MTM outperforms themedian filter in removing additive noise at the cost of increasedcomputational complexity, but not as well as the median filter inremoving impulse noise. The α-TM filter is in general superior to themedian filter as an impulse detector, but its performance in removingimpulse noise is not so well. As is shown in the test example of thispaper, the α-TM filter performs even worse than the median filter whenno impulse detection techniques are employed. Nevertheless, when thelevel of impulse noise is high, the α-TM filter is not optimal fordetection, since the selected pixels may have a large probability ofbeing corrupted by impulse noise. Removal of additive noise is a problemfor the α-TM and MTM filters because of not taking account of thecentral pixel in the window.

A new filtering technique is disclosed, using what we call a generalizedtrimmed mean (GTM) filter, which in general outperforms (α-TM and MTMfilters for images that are highly corrupted by impulse or additivenoise. The GTM filter is based on a generalization of the α-TM filter.In addition to the commonly used moving spatial window Φ(m,n), asymmetric median basket is employed to collect a predetermined number ofpixels on both sides of the median value of the sorted pixels in thewindow. The luminance values of the collected pixels and the centralpixel in the window are weighted and averaged to obtain an adjustedvalue G(m,n) for the central pixel (m,n). For the removal of additivenoise, it is very useful to have the central pixel participate in theaveraging operation because the probability of the luminance value ofthe central pixel to be the closest to the noise-free value is largerthan for the other pixels. However, when impulse noise is presence, weusually set the weight of the central pixel to zero because theimpulse-noise-corrupted pixels are independent to the noise-free pixels.For images that are highly corrupted with impulse noise, it is ingeneral effective to require the selected values in the median basket tohave different weights in order that the impulses selected for thebasket not affect the filtering output very much. Furthermore, in theprocedure of impulse noise removal, the proposed filter is combined witha switching scheme [9] to produce an impulse detector to preserve thenoise-free pixels exactly, while providing an optimal approximation forthe noise-corrupted pixels. Many impulse detection algorithms have beenproposed [9-12]. In [9], a median filter-based switching impulsedetector is employed. The basic idea is to calculate the absolutedifference between the median filtered value and the input value foreach pixel. If the difference is larger than a given threshold, theoutput is the median filtered value; otherwise, the output equals theinput value. We do the same except that we replace the median filterwith the GTM filter. In order to remove impulse noise effectively, wehere propose a new iteration scheme which generally improves theperformance of the filter. We use both the output of the last iterationand the initial input as the input in the current iteration calculation.The GTM filter is applied to the output of last iteration to give anintermediate output. If the absolute difference between the initialinput and the intermediate output is larger than a predeterminedthreshold T, the current output is the intermediate output, otherwise,the current output is the initial input. In contrast to the traditionaliteration technique, our iteration scheme does not use the output of thelast iteration to do the switching operation (the second step).

GENERALIZED TRIMMED MEAN FILTER

In the commonly-used median filtering procedure, the luminance value ofa pixel is replaced by the median value in a neighboring spatial windowΦ(m,n)=([m−W, m+W]×[n−W, n+W])   (170)The size of this moving square window is thus N=(2W+1)×(2W+1). Let themedian luminance value in this spatial window be denoted asM(m,n)=Median ({I(i,j)}|I(i,j))∈Φ(m,n)   (171)where I(i,j) is the luminance value at pixel (i,j). We reorganize thepixels in the window as a new list according to the ascending order oftheir luminance valuesΓ(−└(N−1)/2┘)≦ . . . ≦Γ(−1)≦Γ(0)≦Γ(1) ≦ . . . ≦Γ(└(N+1)/2┘)  (172)where Γ(0) is the median value in the neighborhood of pixel (m,n), i.e.,Γ(0)=M(m,n).

The key generalization to median filtering introduced in thealpha-trimmed mean (α-TM) filter [5-6] is to design a symmetric medianbasket according to luminance value in order to combine a given numberof pixels on both sides of the median value of the sorted pixels in thewindow. The collected pixels are then averaged to give the filteringoutput, as follows $\begin{matrix}{{A( {m,n} )} = {\frac{1}{{2\quad L} + 1}{\sum\limits_{i = {- L}}^{L}{I^{\prime}(i)}}}} & (173)\end{matrix}$where L=└αN┘ with 0≦α≦0.5. When α=0, it becomes the median filter, andwhen α=0.5, it becomes the simple moving average. In general, the α-TMfilter outperforms the median filter in detecting the impulse. However,its capability of removing the impulse noise is even worse than for themedian filter when no additional procedure is employed (such as theswitching scheme). When an image is corrupted by high levels of impulsenoise, the α-TM filter does not perform well, since the pixels beingselected for the median basket now have a large probability of beingimpulse corrupted. It is therefore unreasonable for the α-TM filter tohave the pixels in the basket equally weighted. Another generalizationto the median filtering is the so called modified trimmed mean (MTM)filter [6]. In the MTM filter, a container C(m,n) is employed to selectthe pixels whose luminance values are in the range of [M(m,n)−q,M(m,n)+q], with q being a predetermined threshold. The mean value of theselected values in the container is taken as the filtering output.T(m,n)=Mean ({I(ij)}|I(i,j)∈C(m,n)   (174)Like the α-TM filter, the MTM filter can also be reduced to the medianfilter (at q=0) or the simple moving average (at q=255 for 8 bpp grayscale image). The MTM filter is useful for removing additive noise butdoes not perform as well as the median filter when impulse noise removalis required, since impulse noise corrupted pixels are independent of thenoise-free pixels.

There is still one problem left for the α-TM and MTM filters whereadditive noise removal is concerned; they do not take special account ofthe luminance value of the central pixel in the window. As is wellknown, the value of the central pixel, in general, has a largerprobability of being the closest to its true value than those of allother pixels in the window. We therefore design the generalized trimmedmean (GTM) filter which improves the performance of both the α-TM andMTM filters. In the first step of the new filter in process, we employthe median basket to collect those pixels whose luminance values areclose to the median value, in the same way as is done in the α-TMfilter. The difference between our algorithm and the α-TM algorithm isthat a weighted averaging is performed using the values selected for themedian basket, as well as the value of the central pixel (In general,the central pixel is used in removing the additive noise.). Thus,$\begin{matrix}{{G( {m,n} )} = \frac{{w_{c}{I( {m,n} )}} + {\sum\limits_{i = {- L}}^{L}{w_{i}{I^{\prime}(i)}}}}{w_{c} + {\sum\limits_{i = {- L}}^{L}w_{i}}}} & (175)\end{matrix}$where w_(c) and w_(i)'s are the weights for the central pixel and thepixels in the median basket respectively. When w_(c)32 0 and all w_(i)'sare the same, it becomes the α-TM filter. Where filtering impulse noisecorrupted images is concerned, we usually set w_(c)=0 because theamplitude and the position of the impulse is independent of the originaltrue signal. Compared with the α-TM filter, the GTM filter is usuallymore efficient when highly impulse noise corrupted images are to befiltered. For such an image, the weight w₀ for the median value isusually chosen to be higher than those of other pixels in the medianbasket because the median pixel usually has the least probability to becorrupted by impulse noise. Smaller weights for the pixels other thanthe median pixel can serve as an adjustment for the filtering output,which is important in the iteration calculation. The problems ofapplying the α-TM filter to highly impulse noise corrupted images isthat the weights for the pixels other than the median pixel are toolarge, which may give filtering results that are still highly corrupted.As explained above, because the GTM filter includes the central pixel inthe averaging operation, it is reasonable to expect that its performancein removing additive noise will be improved compared to the α-TM and MTMfilters.

For the removal of impulse noise, a switching scheme [9] based on ourGTM filter is employed to detect the impulse noise corrupted pixels. Thefiltering output I(m,n) for a pixel (m,n) is generated by the followingalgorithm: $\begin{matrix}{{I( {m,n} )} = \{ \quad\begin{matrix}{{I_{i}( {m,n} )},} & {{{{I_{i}( {m,n} )} - {G( {m,n} )}}} < T} \\{{G( {m,n} )},} & {{{{I_{i}( {m,n} )} - {G( {m,n} )}}} \geq T}\end{matrix}\quad } & (176)\end{matrix}$where I_(i) is the original input pixel value and T is the switchingthreshold used to test the difference between the original input andoutput of the GTM filter.

a) If the difference is larger than T, it implies that the pixel differssignificantly from its neighbors. It is therefore identified as a noisecorrupted pixel, and is replaced by G(m,n).

b) If the difference is smaller than T, it implies the original pixel issimilar to its statistical neighbors, and we identify it as noise free,therefore retaining its original input value.

For more seriously impulse-noise-corrupted images, an iterativeapplication of the above procedure is used to improve the filteringperformance. The iteration processing can be summarized as$\begin{matrix}{{I( {m, n \middle| t } )} = \{ \quad\begin{matrix}{{I_{i}( {m,n} )},} & {{{{I_{i}( {m,n} )} - {G( {m,n} )}}} < T} \\{ {{G( m }, n \middle| t } ),} & {{{{I_{i}( {m,n} )}} - {G( {m,{n t )}} }} \geq T}\end{matrix}\quad } & (177)\end{matrix}$where I(m,n|t) is the system output at time t, G(m,n|t) is theintermediate output obtained by applying [6] to I(i,j|t−1) with(i,j)∈Φ(m,n). To initialize the algorithm, we set I(i,j|0)=I_(i)(i,j).The above procedure can be expressed compactly as I(m,n)|t)=I _(i)(m,n)S(m,n)|t)+G(m,n)|t)[1−S(m,n)|t)]  (178)where the step function S(m,n|t) is defined by $\begin{matrix}{{S( {m, n \middle| t } )} = \{ \quad\begin{matrix}{1,} & {{{{I_{i}( {m,n} )} - {G( {m, n \middle| t } )}}} < T} \\{0,} & {{{{I_{i}( {m,n} )} - {G( {m, n \middle| t } )}}} \geq T}\end{matrix}\quad } & (179)\end{matrix}$It is important for the iteration procedure to always compare G(m,n|t)with the initial input I_(i)(m,n) and update the filtering output withI_(i)(m,n) when their difference is less than the threshold T. It is notgood to use I(m,n|t−1) instead of I_(i)(m,n) to either do the comparisonor updating operation.

EXAMPLE APPLICATIONS

To test the proposed filtering technique, the benchmark 8 bpp gray-scaleimage, “Lena”, size 512×512, is corrupted with different percentages offixed value impulse noise (40% and 60%), and the Gaussian noise withpeak signal-to-noise ratio (PSNR) 18.82 dB respectively. The symmetricmoving window size is 3×3, with a 3-entry median basket used for bothcases. The PSNR, mean square error (MSE) and mean absolute error (MAE)comparisons of several different filtering algorithms are shown in TABLE4 for impulse noise and TABLE 5 for Gaussian noise.

TABLE 4 Comparative Filtering Results for Lena Image with Fixed ImpulseNoise Impulse (40%) Impulse (60%) PSNR PSNR Algorithm* (dB) MSE MAE (dB)MSE MAE No Denoising 9.29 7659.43 51.03 7.54 11467.21 76.45 Median (3 ×3) 28.75 86.79 4.72 23.38 298.71 7.25 Median (5 × 5) 28.15 99.65 5.2126.15 157.65 6.54 α-TM (3 × 3) 27.49 115.86 6.05 23.44 294.41 10.70Median Switch 30.29 60.77 2.54 23.73 275.26 5.50 (3 × 3) [5] α-TM Switch30.72 55.15 2.33 26.66 140.31 4.52 (3 × 3) GTM Switch 31.18 49.55 2.2227.57 113.84 3.98 (3 × 3) *All algorithms are implemented recursivelyfor optimal PSNR and MSE performance. All switching algorithms areiterated in the same way and the switching threshold T is 28.Three-entry median basket are used in the α-TM and the new algorithms.The weights for our algorithm are w⁻¹:w₀:w₁:w_(c) = 1:14:1:0.

TABLE 5 Comparative Filtering Results for Lena Image with Gaussian NoisePSNR Algorithm (dB) MSE MAE No Denoising 18.82 853.07 23.71 Median (3 ×3) 27.60 112.97 7.72 Median (5 × 5) 27.39 118.41 7.53 MTM (3 × 3) 28.2397.74 7.39 α-TM (3 × 3) 28 13 100.10 7.33 GTM (3 × 3) 28.38 94.48 7.04*All algorithms are implemented recursively for optimal PSNR and MSEperformance. Three-entry median basket are used in the last twoalgorithms. The weights for our algorithm are w⁻¹:w₀:w₁:w_(c) =1:1:1:10. The threshold q for the MTM algorithm is optimized to 100.

The filtering parameters are also shown in the TABLES 4 and 5. Allalgorithms are implemented recursively for optimal PSNR and MSEperformance. The new iteration scheme proposed in this disclosure alsohas been implemented with α-TM filter for filtering of impulse noise. Itis evident from TABLE 4 that our GTM filter-based switching schemeyields improved results compared to both the median and α-TMfilter-based switching schemes. In general, the α-TM filter-basedswitching scheme performs better than the median filter-based switchingscheme. However, without the switching scheme, the α-TM filter performseven worse than the median filter for such highlyimpulse-noise-corrupted images, showing that although the α-TM filter isa good impulse detector, it is not good at removing impulse noise byitself. As shown in FIG. 55(a), the original Lena image is degraded byadding 60% impulse noise. FIG. 55(b) is the filtering result using themethod of Sun and Neuvo [9] with a 3×3 window and switching threshold ofT=28. One can observe both blur and speckles on the image; the 3×3window is not suitable for applying the median filtering algorithms forsuch a high-noise image. These speckles can be removed by increasing thewindow size from 3×3 to 5×5 but at the expense of even more blurringafter many iterations. FIG. 55(c) shows our filtering result with thesame switching threshold. It is seen that our algorithm yields improvedresults. Because our algorithm is implemented in a 3×3 window, it ispossible to preserve more detail of the image than the algorithm of Sunand Neuvo [9] implemented in a 5×5 window with many iterations. Theremaining small number of speckles can be removed by filtering using a5×5 window once (with the same basket-weight vector) and then continuingto iterate using the 3×3 window; see FIG. 55(d). For the 40% impulsenoise corrupted Lena image in FIG. 56(a), the filtering performance iscompared with commonly used median filters having different window size.In order to remove the speckles from the image, the median filters mustintroduce some blurring and aliasing to the image (FIGS. 56(b) and56(c)), while our filtering result is significantly improved (FIG.56(d)) in comparison. For the filtering of highlyimpulse-noise-corrupted images, the performance is usually improved whenthe median value is given a larger weight than other pixels in themedian basket. TABLE 5 shows that it is reasonable to include theluminance value of the central pixel in the averaging operation for theremoving of Gaussian noise. With a weight of the central pixel largerthan those of other pixels in the median basket, improving iterationperformance can be obtained in removing additive noise.

CONCLUSIONS

In this disclosure, we present a new algorithm for image- noise removal.A given number of significant luminance values on both sides of themedian value of the sorted pixels in the neighboring window are bundledwith the central pixel and weighted to obtain a modified luminanceestimate (MLE) for the central pixel. For the removal of impulse noise,a threshold selective-pass technique is employed to determine whetherthe central pixel should be replaced by its MLE. A new iterativeprocessing is designed to improve the performance of our algorithm forhighly impulse noise corrupted images. For effectively removing additivenoise, it is useful for the filtering technique to include the value ofthe central pixel as one of the values being averaged. Numericalexperiments show that our technique is simple, robust and efficient, andleads to significant improvement over other well-known methods.

REFERENCES

-   [1] A. Rosenfeld and A. C. Kak, Digital Picture Processing, New    York: Academic Press, Vol. 1, 1982.-   [2] H. M. Lin and A. N. Willson, “Median filters with adaptive    length,” IEEE Trans. Circuits Syst., Vol. 35, pp. 675-690, June    1988.-   [3] D. R. K. Brownrigg, “The weighted median filter,” Comm. Assoc.    Comput. Mach., Vol. 27, pp. 807-818, 1984.-   [4] S.-J. Ko and Y. H. Lee, “Center weighted median filters and    their applications to image enhancement,” IEEE Trans. Circuits    Syst., Vol. 38, pp. 984-993, September 1991.-   [5] J. B. Bednar and T. L. Watt, “Alpha-trimmed means and their    relationship to median filters,” IEEE Trans. Acoust. Speech, Signal    Processing, Vol. ASSP-32, pp. 145-153, 1984.-   [6] Y. H. Lee and S. A. Kassam, “Generalized median filtering and    related nonlinear filtering techniques,” IEEE Trans. Acoust.,    Speech, Signal Processing, Vol. ASSP-33, pp. 672-683, 1985.-   [7] S. Peterson, Y. H. Lee, and S. A. Kassam, “Some statistical    properties of the alpha-trimmed mean and standard type M filters,”    IEEE Trans. Acoust., Speech, and Signal Processing, Vol. ASSP-36,    pp. 707-713, 1988.-   [8] W.-R. Wu and A. Kundu, “A new type of modified trimmed mean    filter,” in Nonlinear Image Processing II., E. R. Dougherty, G. R.    Arce, and C. G. Boncelet, Jr., Eds. San Joes, Calif.: SPIE, 1991,    Vol. 1451, pp. 13-23.-   [9] T. Sun, Y. Neuvo, “Detail-preserving median based filters in    image processing,” Pattern Recognition Letter, Vol. 15, pp. 341-347,    April 1994.-   [10] R. Sucher, “Removal of impulse noise by selective filtering”,    IEEE Proc. Int. Conf. Image Processing, Austin, Tex., Nov. 1994, pp.    502-506.-   [11] E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, “A new    efficient approach for the removal of impulse noise from highly    corrupted images,” IEEE Trans. Image Processing, vol. 5, pp.    1012-1025, June 1996.-   [12] Z. Wang, D. Zhang, “Restoration of impulse noise corrupted    images using long-range correlation,” IEEE Signal Processing Letter,    Vol. 5, pp. 4-7, Jan. 1998.

BIOMEDICAL SIGNAL PROCESSING USING A NEW CLASS OF WAVELETS Introduction

In general, signal filtering and processing may be regarded as a kind ofapproximation problem with noise suppression. According to DAF theory[1], a signal approximation in DAF space can be expressed as$\begin{matrix}{{\hat{g}(x)} = {\sum\limits_{i}\quad{{g( x_{i} )}{\delta_{\alpha}( {x - x_{i}} )}}}} & (180)\end{matrix}$where the δ_(α)(x) is a generalized symmetric Delta functional sequence.We choose it as a Gauss modulated interpolating shell, or the so-calleddistributed approximating functional (DAF) wavelet. The Hermite-type DAFwavelet is given in the following equation [2]. $\begin{matrix}{{\delta_{M}( x \middle| \sigma )} = {\frac{1}{\sigma}{\exp( \frac{- x^{2}}{2\sigma^{2}} )}{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2\pi}{n!}}{H_{2n}( \frac{x}{\sqrt{2}\sigma} )}}}}} & (181)\end{matrix}$The function H_(2n) is the Hermite polynomial of even order, 2n. Thequalitative behavior of one particular Hermite DAF is shown in FIG. 57.

Different selections of interpolating shells result in different DAFs.Theoretically, this kind of functional can be regarded as the smoothingoperator or the scaling function in wavelet theory. It can be used togenerate the corresponding wavelets (differential functionals) forsignal analysis. The discrete wavelet transform is implemented usingfilterbanks.

Additionally, the continuous type of DAF wavelet is used to constructDAF wavelet neural nets. The DAF wavelet neural nets possess a modifiedform the commonly used DAF approximation, given as $\begin{matrix}{{\hat{g}(x)} = {\sum\limits_{i}{{w(i)}{\delta_{\alpha}( {x - x_{i}} )}}}} & (182)\end{matrix}$The weights w(i) of the nets determine the superposition approximationĝ(x) to the original signal g(x)∈L²(R). It is easy to show that theweights of the approximation nets, w(i), are closely related to the DAFsampling coefficients g(x_(i)). If the observed signal is limited to aninterval I containing a total of N discrete samples, I={0, 1, . . . ,N−1}, the square error of the signal is digitized according to$\begin{matrix}{E_{A} = {\sum\limits_{n = 0}^{N - 1}\quad\lbrack {{g(n)} - {\hat{g}(n)}} \rbrack^{2}}} & (183)\end{matrix}$This cost function is commonly used for neural network training in anoise-free background and is referred to as the minimum mean squareerror (MMSE) rule. However, if the observed signal is corrupted bynoise, the network produced by MMSE training causes an unstablereconstruction, because the MMSE recovers the noise components as wellas the signal. MMSE may lead to Gibbs-like undulations in the signal,which is especially harmful for calculating accurate derivatives. Thus,we present a novel regularization design of the cost function fornetwork training. It generates edge-preserved filters and reducesdistortion. To achieve this, an additional smooth derivative term,E_(r), is introduced to modify the original cost function. The new costfunction is $\quad\begin{matrix}{E = {{E_{A} + {\lambda\quad E_{r}}} = {{\sum\limits_{k}\lbrack {{g(k)} - {\hat{g}(k)}} \rbrack^{2}} + {\lambda{\int_{R}{\lbrack \frac{\partial^{\prime}{\hat{g}(x)}}{\partial x^{\prime}} \rbrack^{2}\quad{\mathbb{d}x}}}}}}} & (184)\end{matrix}$The factor λ introduces a compromise between the orders of approximationand smoothness. To increase the stability of the approximation systemfurther, an additional constraint in state space is taken to be$\begin{matrix}{E_{W} = \frac{\sum\limits_{i}\quad{{w(i)}}^{2}}{\sum\limits_{i}\quad{{g( x_{i} )}}^{2}}} & (185)\end{matrix}$Thus the complete cost function utilized for DAF wavelet neural nettraining is given by $\begin{matrix}{E = {{E_{A} + {\lambda\quad E_{r}} + {\eta\quad E_{w}}} = {{\sum\limits_{k}\quad\lbrack {{g(k)} - {\hat{g}(k)}} \rbrack^{2}} + {\lambda{\int_{R}{\lbrack \frac{\partial^{\prime}{\hat{g}(x)}}{\partial x^{\prime}} \rbrack^{2}\quad{\mathbb{d}x}}}} + {\eta\quad\frac{\sum\limits_{i}\quad{{w(i)}}^{2}}{\sum\limits_{i}\quad{{g( x_{i} )}}^{2}}}}}} & (186)\end{matrix}$

MAMMOGRAM ENHANCEMENT

Medical imaging, including computed tomography (CT), magnetic resonanceimaging (MRI), X-ray mammography, ultrasound and angiography, includessome of the most useful methods for diagnosis. In particular, X-raymammograms are widely recognized as being the most effective method forthe early detection of breast cancer. Major advances in mammograms havebeen made over the past decade, which have resulted in significantimprovements in image resolution and film contrast without much increasein X-ray dosage. In fact, mammogram films have the highest resolution incomparison of various other screen/film techniques. However, many breastcancers cannot be detected based on mammogram images because of the poorvisualization quality of the image. There are also many false positivesthat result in substantial stress to patients and their families. Bothtypes of difficulties are substantially due to the minor differences inX-ray attenuation between normal glandular tissues, benign formationsand malignant disease, which leads to the low-contrast feature of theimage. As a result, early detection of small tumors, particularly foryounger women who have denser breast tissue, is still extremelydifficult.

Mammogram image processing has recently drawn a great deal of attention[3-8]. Most work focuses on noise reduction and feature enhancement. Thestatistical noise level is relatively low for images obtained by moderndata acquisition techniques. Therefore feature enhancement is moreessential for mammogram quality improvement. Since an image noisereduction algorithm was reported earlier [9, 10], we shall focus onimage enhancement in this work. In our opinion, there are two basicapproaches for mammogram feature improvement. One type of method changesthe image spectrum distribution so as to increase image edge densities.As a result, the image appears sharper. The other approach is theso-called image windowing, in which linear or nonlinear grayscalemappings are defined so as to increase/compress the image gradientdensity. In an earlier work, one of present authors realized that thefrequency responses of wavelet transform subband filters are notuniform. A magnitude normalization algorithm was proposed to account forthis non-uniform feature. This idea was later extended to groupnormalization for the wavelet packet transform [11, 12] and to visualgroup normalization for still-image noise reduction [13] by the presentauthors. In this work we further extend this idea to achieve imageenhancement.

Our first approach is based on a rearrangement of the image spectrumdistribution by an edge enhancement normalization of the waveletcoefficients. Our second technique utilizes a multiscale functional toobtain a device-adapted visual group normalization of the waveletcoefficients. Numerical experiments indicate that our normalizationapproach, in combination with the use of the DAF wavelets [13], providesexcellent enhancement for low quality mammogram images.

To test our new approaches, low-contrast and low quality breastmammogram images are employed. Mammograms are complex in appearance andsigns of early disease are often small or subtle. Digital mammogramimage enhancement is particularly important for aiding radiologists inlong distance consultation, image storage and retrieval, and for thepossible development of automatic detecting expert systems. FIG. 58(a)is an original 1024×1024 side-view breast image which was obtained fromthe Mammographic Image Analysis Society (MIAS) and has been digitized to8 bits of gray scale. The enhanced image result is depicted in FIG.58(b), and shows significant improvement in image quality.

ECG FILTERING

Automatic diagnosis of electrocardiogram (ECG or EKG) signals is animportant biomedical analysis tool. The diagnosis is based on thedetection of abnormalities in an ECG signal. ECG signal processing is acrucial step for obtaining a noise-free signal and for improvingdiagnostic accuracy. A typical raw ECG signal is given in FIG. 59. Theletters P, Q, R, S, T and U label the medically interesting features.For example, in the normal sinus rhythm of a 12-lead ECG, a QRS peakfollows each P wave. Normal P waves show 60-100 bpm with <10%variations. Their heights are <2.5 mm and widths <0.11 s in lead II. Anormal PR interval ranges from 0.12 to 0.20 s (3-5 small squares). Anormal QRS complex has a duration of <0.12 s (3 small squares). Acorrected QT interval (QTc) is obtained by dividing the QT interval withthe square root of the preceding R-R′ interval (normally QTc=0.42 s). Anormal ST segment indicates no elevation or depression. Hyperkalaemia,hyperacute myocardial infarction and left bundle can cause an extra tallT wave. Small, flattened or inverted T waves are usually caused byischaemia, age, race, hyperventilation, anxiety, drinking iced water,LVH, drugs (e.g. digoxin), pericarditis, PE, intraventricular conductiondelay (e.g. RBBB) and electrolyte disturbance [20].

An important task of ECG signal filtering is to preserve the truemagnitudes of the P, Q, R, S, T, and U waves, protect the true intervals(starting and ending points) and segments, and suppress distortionsinduced by noise. The most common noise in an ECG signal is ACinterference (about 50 Hz-60 Hz in the frequency regime). Traditionalfiltering methods (low-pass, and band-elimination filters, etc.)encounter difficulties in dealing with the AC noise because the signaland the noise overlap the same band. As a consequence, experienceddoctors are required to carry out time-consuming manual diagnoses.

Based on a time varying processing principle, a non-linear filter [14]was recently adopted for ECG signal de-noising. Similar to the selectiveaveraging schemes used in image processing, the ECG is divided intodifferent time segments. A sample point classified as “signal” issmoothed by using short window averaging, while a “noise” point istreated by using long window averaging. Window width is chosen accordingto the statistical mean and variance of each segment. However, thiscalculation is complicated and it is not easy to select windows withappropriate lengths. The regularized spline network and wavelet packetswere later used for adaptive ECG filtering [12, 15], which is not yetefficient and robust for signal processing. In our present treatment,regularized DAF neural networks are used to handle a real-valued ECGsignal. We utilize our combined group-constraint technique to enhancesignal components and suppress noise in successive time-varying tilings.

The raw ECG of a patient is given in FIG. 60(a). Note that it hastypical thorn-like electromagnetic interference. FIG. 60(b) is theresult of a low-pass filter smoothing. The magnitudes of the P and Rwaves are significantly reduced and the Q and S waves almost disappearcompletely. The T wave is enlarged, which leads to an increase in the QTinterval. Notably, the ST segment is depressed. Such a low-passfiltering result can cause significant diagnostic errors. FIG. 60(c) isobtained by using our DAF wavelet neural nets. Obviously, our methodprovides better feature-preserving filtering for ECG signal processing.

REFERENCES

-   [1] D. K. Hoffman, G. W. Wei, D. S. Zhang, D. J. Kouri,    “Shannon-Gabor wavelet distributed approximating functional,”    Chemical Phyiscs Letters, Vol. 287, pp. 119-124, 1998.-   [2] D. S. Zhang, G. W. Wei, D. J. Kouri, and D. K. Hoffman,    “Numerical method for the nonlinear Fokker-Planck equation,” Phys.    Rev. E, Vol. 56, No. 1, pp. 1197-1206, 1997.-   [3] A. P. Dhawan and E. Le Royer, “Mammographic feature enhancement    by computerized image processing,” Comput. Methods and Programs in    Biomed., vol. 27, no. 1, pp. 23-35, 1988.-   [4] L. Gagnon, J. M. Lina, and B. Goulard, “Sharpening enhancement    of digitized mammograms with complex symmetric Daubechies wavelets,”    preprint.-   [5] S. Lai, X. Li, and W. F. Bischof, “On techniques for detecting    circumscribed masses in mammograms,” IEEE Trans. Med. Imag., vol. 8,    no. 4, pp. 377-386, 1989.-   [6] A. F. Laine, S. Schuler, J. Fan, and W. Huda, “Mammographic    feature enhancement by multiscale analysis,” IEEE Trans. Med. Imag.,    vol. 13, no. 4, 1994.-   [7] J. Lu, D. M. Healy Jr., and J. B. Weaver, “Contrast enhancement    of medical images using multiscale edge representation,” Optical    Engineering, in press.-   [8] P. G. Tahoces, J. Correa, M. Souto, C. Gonzalez, L. Gomez,    and J. J. Vidal, “Enhancement of chest and 7breast radiographs by    automatic spatial filtering,” IEEE Trans. Med. Imag., vol. 10, no.    3, pp. 330-335, 1991.-   [9] M. Nagao and T. Matsuyama, “Edge preserving smoothing,” Comput.    Graphics and Image Processing, vol. 9, no. 4, pp. 394-407, 1979.-   [10] A. Scheer, F. R. D. Velasco, and A. Rosenfield, “Some new image    smoothing techniques,” IEEE Trans. Syst., Man. Cyber., vol. SMC-10,    no. 3, pp. 153-158, 1980.-   [11] Z. Shi and Z. Bao, “Group-normalized wavelet packet signal    processing,” Proc. SPIE, Vol. 3078, pp. 226-239, 1997.-   [12] Z. Shi, Z. Bao, L. C. Jiao, “Nonlinear ECG filtering by group    normalized wavelet packets”, IEEE International Symposium on    Information Theory, Ulm, Germany, 1997-   [13] Z. Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, “Perceptual    normalized subband image restoration,” IEEE Symp. On Time-frequency    and Time-scale Analysis, N. 144, pp. 469-472, Pittsburgh, Pa., Oct.    6-9, 1998.-   [14] Z. Shi, “Nonlinear processing of ECG signal,” B.Sc. Thesis,    Xidian Univ., 1991.-   [15] Z. Shi, L. C. Jiao, Z. Bao, “Regularized spline networks,”    IJCNN'95, Beijing, China. (also J. Xidian Univ., Vol. 22, No. 5, pp.    78˜86, 1995.)-   [16] M. Unser, A. Aldroubi, and M. Eden, “B-spline signal    processing: part I—theory,” IEEE Trans. SP, Vol. 41, No. 2, pp.    821-833, February 1993-   [17] M. Unser, A. Aldroubi, and M. Eden, “B-spline signal    processing: part II—Efficient design and applications,” IEEE Trans.    SP, Vol. 41, No. 2, pp. 834˜848, February 1993.-   [18] A. B. Watson, G. Y. Yang, J. A. Solomon and J. Villasenor,    “Visibility of wavelet quantization noise,” IEEE Trans. Image    Processing, vol. 6, pp. 1164-1175, 1997.-   [19] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,    “Lagrange distributed approximating Functionals,” Phys. Rev. Lett.,    Vol. 79, No. 5, pp. 775-779, 1997.-   [20] The reader is referred to the website    homepages-enterprise-net/djenkins.

VISUAL GROUP NORMALIZATION USING GAUSSIAN-LAGRANGE DAFWs Introduction

Distributed approximating functionals (DAFs) were introduced as apowerful grid method for quantum dynamical propagations [1]. DAFs can beregarded as scaling functions and associated DAF-wavelets are generatedin a number of ways [2]. DAF-wavelets are smooth and decaying in bothtime and frequency domains and have been used for numerically solvinglinear and nonlinear partial differential equations with extremely highaccuracy and computational efficiency. Typical examples includesimulations of 3D reactive quantum scattering and 2D Navier-Stokesequation with non-periodic boundary conditions. The present work extendsthe DAF approach to image processing by constructing interpolatingDAF-wavelets[3]. An earlier Group normalization (GN) technique [4] andhuman vision response [5] are utilized to normalize the equivalentdecomposition filters (EDFs) and perceptual luminance sensitivity. Thecombined DAF Visual Group Normalization (VGN) approaches achieve robustimage restoration result.

INTERPOLATING DAF WAVELETS

Interpolating wavelets are particularly efficient for calculation sincetheir multiresolution spaces are identical to the discrete samplingspaces. Adaptive boundary treatments and irregular samplings can beeasily implemented using symmetric interpolating solutions. We designthe interpolating scaling function as an interpolating Gaussian-LagrangeDAF (GLDAF) [6] $\quad\begin{matrix}{{\phi_{M}(x)} = {{{W_{\sigma}(x)}{P_{M}(x)}} = {{W_{\sigma}(x)}\quad{\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\quad\frac{x - i}{- i}}}}} & (187)\end{matrix}$where W_(σ)(x) is selected as a Gaussian window since it satisfies theminimum-frame-bound condition in quantum physics. Quantity σ is widthparameter. P_(M)(x) is the Lagrange interpolation kernel. DAF dualscaling and wavelet functions are constructed for perfect reconstruction[3]. The Gaussian window efficiently smoothes out the oscillations,which plague many wavelet bases.

VISUAL GROUP NORMALIZATION

Wavelet transform is implemented by a tree-structure filteringiteration. The coefficients can be regarded as the output of a singleequivalent decomposition filters (EDF). Clearly, the decompositioncoefficients cannot exactly represent the actual signal strength. As anadjusted solution, wavelet coefficients, C_(m)(k), in block m shouldmultiply with a magnitude scaling factor, λ_(m). We define this factoras the reciprocal of the maximum magnitude of the EDF frequency response[4] (where LC_(m) is m-th EDF response). $\begin{matrix}{{\lambda_{j,m} = \frac{1}{\sup\limits_{\omega \in \Omega}\quad\{ {{{LC}_{j,m}(\omega)}} \}}},\quad{\Omega = \lbrack {0,{2\quad\pi}} \rbrack}} & (188)\end{matrix}$

It is easy to find out that DAF's possess smaller sidelobes, whichinduce less frequency leakage distortion.

An image is human-vision-dependent source. Using a just-noticeabledistortion profile, we can efficiently remove the visual redundancy aswell as normalize the coefficients in respect to perception importance.A mathematical model for “perception lossless” matrix Y_(m) has beenpresented in [5] and is used as perceptual normalization combined withthe EDF magnitude normalization. (Noted here we use λ_(m) for magnitudenormalization not the wavelet “basis function amplitude” in [5], becausethe digital image decomposition is completely operated using filterbanks) We denote the combination of the two normalized operations asVisual Group Normalization (VGN).

EXPERIMENTAL RESULTS

To test our approaches, two benchmark 512×512 Y-component images areemployed (Barbara with much texture and high frequency edges, while Lenawith large area of flat region). The PSNR comparison results are shownin Table 3. It is evident that our technique yields better performance.

REFERENCES

-   [1] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri,    “Analytic banded approximation for the discretized free    propagator,” J. Physical Chemistry, vol. 95, no. 21, pp. 8299-8305,    1991-   [2] Z. Shi, D. J. Kouri, G. W. Wei, and D. K. Hoffman, “Generalized    symmetric interpolating wavelets,” Computer Phys. Commun., in press.-   [3] Z. Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, “Perceptual    Normalized Subband Image Restoration,” IEEE Symp. On Time-frequency    and Time-scale Analysis, N. 144, pp. 469-472, Pittsburgh, Pa., Oct.    6-9, 1998.-   [4] Z. Shi, Z. Bao, “Group-normalized wavelet packet signal    processing”, Wavelet Application IV, Proceeding. SPIE, Vol. 3078,    pp. 226-239, Orlando, Fla., USA, 1997-   [5] Andrew B. Watson, Gloria Y. Yang, Joshua A Solomon, and John    Villasenor “Visibility of wavelet quantization noise,” IEEE Trans.    Image Processing, vol. 6, pp. 1164-1175, 1997.-   [6] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,    “Lagrange distributed approximating Functionals,” Physical Review    Letters, Vol. 79, No. 5, pp. 775˜779, 1997-   [7] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.    Information Theory, vol. 41, no. 3, pp. 613˜627, 1995

NONLINEAR QUINCUNX FILTERS Introduction

The possible image noises include photoelectric exchange, photo spots,the error of image communication, etc. The noise causes the speckles,blips, ripples, bumps, ringings and aliasing that not only affects thevisual quality, but also degrades the efficiency of data compression andcoding. Developing de-noising and smoothing technique is important.

The popularly used noise model is Gaussian, since it is easy for linear(stationary) analysis. In practice, the further physical environmentsare more accurately modeled as impulsive, which characterized byheavy-tailed non-Gaussian distributions. Such noises include atmosphericnoise (due to a lightening spike and spurious radio emission in radiocommunication), ice cracking and aquatic animal activity in sonar andmarine communication, and relay switching noise in telephone channels.Moreover, a great variety of synthetic noise sources (such as automaticignitions, neon lights, and other electronic devices) are alsoimpulsive. Impulsive interference may be broadly defined as signalcorruption that is random, sparse, and of high or low amplitude relativeto nearby uncorrupted signal values. The performance of filter systemsdeveloped under the assumption of Gaussian condition are severelydegraded by the non-Gaussian noise due to large deviations fromnormality in the tails. Independent on the physical environment,relatively infrequently occurring and non-stationary are the importantfeatures of impulse noise that we can not obtain an accurate statisticaldescription. Therefore, it is impossible to design an optimal linearfiltering systems based on the generalized likelihood ratio (GLR)principles. Consequently, by first suppressing the impulsive componentand then operating on the modified signal with traditional linearsystem, near-optimal performance is obtained. The non-Gaussian nature ofthe impulse noise dictates that the suppression filter be nonlinear androbust due to the presence of impulses. Traditional image processing isalways defined on the whole space (time) region, which does not localizethe space (time)-frequency details of the signal. The mean error may beimproved, but the averaging process will blur the silhouette and finerdetails.

New research shows that non-Gaussian and non-stationary characterize thehuman-visual-response (HVS) based image processing. Human visualperception is more sensitive to image edges which consist ofsharp-changes of the neighboring gray scale because it is essentiallyadaptive and has variable lenses and focuses for different visualenvironments. To protect edge information as well as remove noise,modern image processing techniques are predominantly based on non-linearmethods. Before the smoothing process, the image edges, as well asperceptually sensitive texture must be detected. Some of the mostpopular nonlinear solutions are the median filter [1] and itsgeneralizations [2, 3], that possess the required properties for impulsenoise removal. However, the median filter is not optimal since it istypically implemented uniformly across the image. It suppresses the truesignal as well as the noise in many applications. In the presence ofimpulse noise, the median filter tends to modify pixels that are notdegraded by noise. Furthermore, it is prone to produce edge jitter whenthe percentage of impulse noise is large.

A significant characteristic of the impulse noise is that only a portionof the pixels are corrupted and the rest are totally impulse-noise-free.The key point for filter design is the need to preserve the noise-freepixels exactly, while providing an optimal approximation for thenoise-corrupted pixels. An impulse detector is required to determine thenoise-corrupted pixels [4-8]. In [4], a median filter-based switchingscheme is used to design the impulse detector. The basic idea is tocalculate the absolute difference between the median filtered value andthe original input value for each pixel. If the difference is largerthan a given threshold, the output is the median filtered value;otherwise, the output is the original input value. However, using asingle median filtered value can not increase the detection probabilityof impulse noise. In addition, median estimation is not optimal forhigh-percentage impulse-degraded image restoration. In this paper, wepropose a so-called “median radius” to design a collecting basket. The“basket members”, whose luminance values are close enough to medianvalue, are weighted averaging to generate an adjustable impulse detector(optimal estimator). Noted here our technique is not the previouslypresented α-trim solutions [2]. For α-trim filtering, the median iscalculated at first, then the pixels in a neighboring window are orderedaccording to the absolute difference from the median value (from minimumto maximum). The first three (or several) pixels (close to median) areweighted averaging to substitute the observed pixel luminance. In ourmethod, we only collect the weighted pixels which are within the “medianradius” (α-trim member may exceed this domain) for averaging.Additionally, a switching threshold is introduced to determine whetherthe observed pixel need filtering or not. Only the noise-corrupted oneis changed.

To improve the filtering performance, a “quincunx moving window” ispresented to define the arbitrary shaped neighborhood (traditionalwindow is the trivial square shape). The research in neurophysiology andpsychophysical studies shows that the direction-selective cortexfiltering mimics the human vision system (HVS). These studies have founda redial frequency selectivity that is essentially symmetric withbandwidths nearly constant at one octave [12]. This feature is much likethe 2-D quincunx decomposition. Theoretical analysis and simulation showthat our quincunx filtering technique is quite robust and efficient fordifferent noise background.

The disclosure is organized as follows: In Section II, the mathematicalmodel of noise is described. In Section III, the image processingtechnique based on a quincunx filter is studied. Section IV concerns thearbitrary shape extension of the quincunx filter. A simulation is givenin Section V. Section VI contains the conclusion.

NOISE MODEL

Additive Random Noise

For a 2-D digital image, we assume the noise-free image value at pixel(m,n) is s(m,n), and I(m,n) is the observed image representation. If theobservation model is a random noise corrupted case, the pixel value canbe depicted as

 I(m,n)=s(m,n)+w(m,n)   (189)

where w(m,n) is the noise components, and its magnitude is random.Normally we assume w(m,n) is a Gaussian process with zero mean andstandard deviation σ_(w). For a non-stationary process, σw istime-varying. The important characteristic of such noise is that allpixels may be degraded and the noise amplitude is statisticallynon-uniform.

IMPULSIVE NOISE

Normally, impulse noise is a result of a random process that the valueof the corrupted pixel is either the minimum or a maximum value of adisplay instrument. The overall noise character could be positive(maximum), negative (minimum) or mixture (salt and pepper). Thecharacteristic of such kind of noise is that the pixels are degradedwith probability p. The noise model is expressed as $\begin{matrix}{{v\quad( {m,n} )} = \{ \begin{matrix}{{e( {m,n} )},} & {{with}\quad p} \\{{I( {m,n} )},} & {{{with}\quad 1} - p}\end{matrix} } & (190)\end{matrix}$where e(m,n) is a binary random number. For an 8-bit gray scale imagerepresentation (0=minima; 255=maxima), its value is $\begin{matrix}{{e\quad( {m,n} )} = \{ \begin{matrix}{0,} & {{with}\quad p_{0}} \\{255,} & {{{with}\quad 1} - p_{0}}\end{matrix} } & (191)\end{matrix}$The probability of the black (minimum) value is p₀ and the probabilityof the white (maximum) value is 1-p₀. Normally for the positive impulsecase, p₀=0; for the negative impulse case, p₀=1; and for the salt andpepper, p₀=½. Any alternation style can be obtained by differentselection of the probability p₀.

More complex impulsive noise models are generated by a random magnitudedegradation selection. The random value impulse noise array ν(m,n) isrepresented asν(m,n)=z(m,n)r(m,n)   (192)where the impulse generation r(m,n) is a random process representing anever-present impulse component with standard deviation σ_(r), and z(m.n)is a switching sequence of ones and zeros. Impulse noise seriouslyaffects the performance of various signal-processing techniques, e.g.edge detection, data compression, and pattern recognition. A medianfilter is the commonly used nonlinear technique for impulse noiseremoval.

Assume we have fixed a median window (possibly with any size) for medianfiltering. Let N_(m,n) denote the number of impulse-corrupted pixels inmedian window Φ(m,n) of node (m,n). If the window size is W(conventionally the 3×3 or 5×5 square window), then it is known that theerror measure is $\begin{matrix}{{E\{ {{x_{med}❘x_{med}} = {e( {m,n} )}} \}} = \frac{E\{ x_{med} \}}{1 - {\Pr\lbrack {x_{med} = {s( {m,n} )}} \rbrack}}} & (193)\end{matrix}$where median estimation probabilityPr[x_(med)=s(m,n)]=Pr[N_(m,n)≦(W−1)/2] is given in [5]. It is easy toconclude that when the corruption probability p is much higher, mostpossibly that N_(m,n)>(W−1)/2. This implies that the convincingprobability (CP) of the median filtering (Pr[x_(med)=s(m,n)]) is verysmall (e.g. with small probability, the output of a median filter isclose to the original noise-free value). In other words, a traditionalmedian filter is invalid for highly noise-corrupted image processing.For a uniformly noise degraded background, no matter what size theselective window is, statistically the performance of median filteringcannot be optimized (because the probability Pr[N_(m,n)≦(W−1)/2] is thefixed statistically). Our objective is to improve the performance ofnonlinear filtering and reduce the error measure above.

NONLINEAR FILTERS

In the median filtering procedure, the luminance value of a pixel isreplaced by the median value in a neighboring spatial square windowΦ(m,n)=([m−m ₁ , m+m ₂ ]×[n−n ₁ , n+n ₂])   (194)The size of this moving rectangular window is N=(m₁+m₂+1)×(n₁+n₂+1). Theconventional trivial windows are 3×3 or 5×5. Let the median luminancevalue in this spatial window be denoted asM(m,n)=Median{I(m′,n′)|(m′,n′)∈Φ}  (195)where I(m′,n′) is the luminance value at pixel (m′,n′). We reorganizethe pixels in the window as a new list according to the order of theirluminance value.I′(−└(N−1)/2┘)≦ . . . ≦ I′(0)≦ . . . ≦I′(N−└(N+1)/2┘)   (196)where I′(0) is exactly the median value in the neighborhood of pixel(m,n),I′(0)=M(m,n)   (197)

The key generalization to median filtering introduced in this disclosureis to design a “basket” according to luminance value in order to combinea group of pixels whose luminance levels are close to the median valueof the window Φ(m,n). For each entry in this basket, a weighted averagescheme is utilized to generate an adjusted median value as$\begin{matrix}{{D( {m,n} )} = \frac{\sum\limits_{i \in \Omega}\quad{W_{i}{I^{\prime}(i)}}}{\sum\limits_{i \in \Omega}\quad W_{i}}} & (198)\end{matrix}$where Ω is the set of pixels whose luminance values are close to medianvalue in the window. The different design of our method from theα-trimmed filter is that we introduce an adjustable basket. The mediandistance (MD) Λ is defined as the absolute luminance difference betweenthe observed value and the median filtered value M(m,n).Λ(m,n)=|I(m,n)−M(m,n)|  (199)Only the pixels whose median distances are within a median radius(MR)γ(e.g. Λ(m,n)≦γ, γ>0), are selected as the “basket member” in Ω forweighted averaging. Otherwise, their weights are set to zero. Normally,W₀ is larger than or equal to the other weights W_(i), i≠0. We callD(m,n) the modified luminance estimate (MLE) for pixel (m,n). Note thatwhen the basket Ω contains only one pixel Γ(0) (or the median radiusγ=0), the filter is identical to the median filter, i.e., the medianfilter is a special case of our presented nonlinear filters.

However, the modified luminance estimate (MLE) is not the ultimatefiltered value to substitute the observed pixel. A switching scheme [2]based on MLE is employed to detect the impulse noise. The correspondingfiltering output I(m,n) for a pixel (m,n) is generated by the followingalgorithm: $\begin{matrix}{{I( {m,n} )} = \{ \begin{matrix}{{I_{i}( {m,n} )},} & {{{{I_{i}( {m,n} )} - {D( {m,n} )}}} < T} \\{{D( {m,n} )},} & {{{{I_{i}( {m,n} )} - {D( {m,n} )}}} \geq T}\end{matrix} } & (200)\end{matrix}$where I_(i) is the original input image. A threshold T is used to testthe difference between the original pixel value and the MLE value. Thedifference between the observed value I(m,n) and the adjusted medianvalue D(m,n) from the basket can be regarded as the visible differencepredictor (VDP).

a) If the difference is larger than the threshold, it implies that thepixel differs significantly from its neighbors. It is thereforeidentified as a noise corrupted pixel, and is replaced by D(m,n).

b) If the difference is smaller than the threshold, it implies theoriginal pixel is similar to its statistical neighbors, and we identifyit as noise free, therefore retaining its original value. For moreseriously noise-corrupted images, an iterative application of the aboveprocedure is required to obtain satisfactory performance. The iterationprocessing can be depicted as $\begin{matrix}{{I( {m,{n❘t}} )} = \{ \begin{matrix}{{I_{i}( {m,n} )},} & {{{{I_{i}( {m,n} )} - {D( {m,{n❘{t - 1}}} )}}} < T} \\{{D( {m,{n❘{t - 1}}} )},} & {{{{I_{i}( {m,n} )} - {D( {m,{n❘{t - 1}}} )}}} \geq T}\end{matrix} } & (201)\end{matrix}$where I(m,n|t) is the system output at time t, D(m,n|t−1) is the MLEvalue of pixel (m,n) at time t-1. To initialize the algorithm, we setI(m,n|0)=I_(i)(m,n). The above procedure can be simply expressed asI(m,n|t)=I _(i)(m,n)S(m,n|t)+D(m,n|t−1)[1−S(m,n|t)],   (202)where the step function S(m,n|t) is defined as $\begin{matrix}{{S( {m,{n❘t}} )} = \{ \begin{matrix}{1,} & {{{{I_{i}( {m,n} )} - {D( {m,{n❘{t - 1}}} )}}} < T} \\{0,} & {{{{I_{i}( {m,n} )} - {D( {m,{n❘{t - 1}}} )}}} \geq T}\end{matrix} } & (203)\end{matrix}$and it determines which value will be assigned to the pixel.

The proposed nonlinear filtering includes two steps; the first step isto obtain the adjusted reference value MLE of the observed pixel usingmedian distance-based algorithm, and the second step is utilizing anactive switching process to determine if the observed pixel issubstituted by the MLE or not. (Noted for the traditional medianfiltering or α-trimmed solution, the pixel is only substituted by thereference value without any switching process.) A special nonlinearprocess is designed to switch between the original pixel and MLE,depending upon the identification of the nature of the pixel. If thepixel is determined as noise-free, the original value should bepreserved. Otherwise, it is replaced by the regressively weighted-medianvalue.

Our filtering is like Olympic scoring procedure, the lowest score andhighest scores (outside the median radius) are removed and the remainingscores are then averaged to obtain the evaluation of the participant. Inour approach, we remove several of the lowest luminance values andseveral of the highest luminance values from the pixels in the window.The remaining luminance values are then weight-averaged to give thefiltering output for the pixel. We may consider the weighted average asassigning different referees different weights in scoring theparticipants.

Theoretically, the MLE generation increases the probabilityPr[x _(med) =s(m,n)]=Pr[N _(m,n)≦(W−1)/2]  (204)When the size of median basket is set to M, it equals to that the medianwindow size W increases M times as MW, but with the same noise corruptednumber N_(m,n). In this case, the detection probability becomesPr[x _(med) =s(m,n)]=Pr[N _(m,n)≦(WM−1)/2]  (205)This implies that the modified filtered output is more likely equal tothe original noise-free value.

QUINCUNX EXTENSION

As shown in FIG. 61, the traditional median or averaging filterconsiders the pixels in a square region. The most popular size is a 3×3or 5×5 window, which are used in the most popular papers and softwarepackets (Photoshop and Lview, etc) on image processing. The filteringmethod or algorithm is applied in the square region (black pixels). Onedisadvantage of square window processing is that it can not adaptive todifferent noisy background and exist biases along the coordinatedirections, or the spatial distance. This may cause visual distortion.

Actually, besides the basic square window, the median basket (or theoperation region) possesses numerous extended solutions. For example,the 5×5 extension is shown in FIG. 62. The traditional square shapeapproach gives preferential treatment to the coordinate axes and onlyallows for rectangular divisions of the frequency spectrum. The symmetryaxes and certain nonrectangular divisions of the frequency spectrumcorrespond better to the human vision system (HVS). These are typicallyconcerned with two and three dimensions, as the algebraic conditions inhigher dimensions become increasingly cumbersome. The predominantadvantages of quincunx filtering is that there is very little biasingalong coordinate directions, the sampling is nearly isotropic; i.e.there is no mixing of different scales; computationally excellentcompression of discontinuous functions; and a simple representation ofoperators. In the 5×5 region, the basket size (black dot number) can be(1, 5, 9, 13, 17, 21, and 25). Thus we will have more alternation tooffset the influence of over-smoothing caused by large window (5×5)processing.

Moreover, some specially designed quincunx windows are conductive toremove the particular noise along different spatial direction, such asthe quantization and thresholding noise of transform filtering andcoding (discrete cosine transform (DCT) and wavelet transform (WT),etc.). Because the high-dimensional transforms (such as 2D DCT andwavelet transform) are usually the tensor products from 1D transform(filtering) along different spatial directions, the coefficient error inthe transform domain causes the distortion in the physical domain withthe shape of the basis function (2D cosine and wavelet function). Itusually looks like a small “cross” impulse. Even for the complicatednon-separable wavelet transform case, the distortion is just a “rotatedcross”. Some quincunx windows will match the exact correlationcharacteristics of transform noise, as well as obtain the optimalperformance. These filters can be utilized as a post-processor after thetransform-based image processing and thresholding techniques. In thenext chapter, we will report our quincunx post-processing results for awavelet thresholding technique.

EXPERIMENTAL RESULTS

We now present the results of computer simulations to demonstrate theeffectiveness of our proposed techniques. Two objective criteria, thesignal-to-noise ratio (SNR) and signal-to-impulse ratio (SIR) are usedto evaluate and compare the performance. The signal-to-noise ratio,which is given by $\begin{matrix}{{SNR} = {10\quad{\log_{10}( \frac{E\{ {s^{2}( {m,n} )} \}}{E\{ \lbrack {{s( {m,n} )} - {\hat{s}( {m,n} )}} \rbrack^{2} \}} )}}} & (206)\end{matrix}$is used to evaluate the overall performance of the proposed preprocessorincluding prediction, nonlinearity and reconstruction capabilities.Another useful quantity, the signal-to-impulse ratio (SIR), is given by$\begin{matrix}{{SIR} = {10\quad{\log\quad}_{10}( \frac{\sigma_{s}^{2}}{\sigma_{v}^{2}} )}} & (207)\end{matrix}$The SNR rule is derived from the traditional image quality criterion,which is characterized by a mean square error (MSE). It possesses theadvantage of a simple mathematical structure. For a discrete signal{s(n)} and its approximation {ŝ(n)}, n=0, . . . , N, the MSE can bedefined to be $\begin{matrix}{{MSE} = {\frac{1}{N}{\sum\limits_{n = 0}^{N - 1}\quad( \lbrack {{\hat{s}(n)} - {s(n)}} \rbrack )^{2}}}} & (208)\end{matrix}$However, the MSE based evaluation standard, such as peak signal-to-noiseratio (PSNR),PSNR=log[(255×255)/MSE]  (209)can not exactly evaluate the image quality if one neglects the effect ofhuman perception. The minimum MSE rule will cause undulations of theimage level and destroy the smooth transition information around thepixels. Therefore, we use both objective and subjective standards toevaluate our presented filtering results.

The benchmark 8bpp gray-scale images are corrupted by additive impulsenoise and Gaussian noise to test the proposed filtering technique. Thepractical symmetric quincunx windows are selected as shown in FIG. 63.The peak signal-noise-ratio (PSNR), mean square error (MSE) and meanabsolute error (MAE) comparison of different filtering algorithms forboth images are shown in TABLE 3 (The amount of noise and the filteringparameters are also shown in TABLE 3.). It is evident that ourfiltering-based switching scheme yields improved results compared to themedian filtering-based switching scheme. In addition, the quincunxextension improves the filtering performance of the nonlinearprocessing. When noise probability is lower (20%), 5-point diamond (orcross) quincunx filter possesses the better performance. When noiseprobability is higher (60%), 13-point diamond quincunx filter istestified better. The 9-point square window is suitable for 40%noise-degraded case. The perceptual quality of our algorithm is shown inFIG. 64. The original Lena image is degraded by adding 40% impulse noisein FIG. 64(a). FIG. 64(b) is the filtering result using our nonlinearfilter.

Wavelet noise is a special kind of additive noise generated bycoefficient quantization error or thresholding error. Because therestored image is regarded as the linear or quincunx combination ofdifferent wavelet functions (scaling and translation), the noise in thephysical domain will be “random wavelets”. For example, in atensor-product wavelet transform case, the wavelet noise has a crossshape. Our quincunx filter can be utilized as a post-processor for awavelet de-noising technique. As shown in FIG. 65, FIG. 65(a) is theGaussian noise-degraded Lena. FIG. 65(b) is the denoising result usingDAF wavelet thresholding technique. FIG. 65(c) is the one-time nonlinearquincunx restoration combined with wavelet thresholding. The result hashigher visual quality and 0.65 dB PSNR improvement.

CONCLUSIONS

In this paper, we present a nonlinear quincunx filter for impulse ormixed noise removal. A specially designed “median basket” based onmedian distance is used to collect the “basket members” for calculatingthe modified median estimate (MLE). A switching scheme is used to detectthe impulse noise and preserve the noise-free pixels. Arbitrary shapequincunx windows are introduced to improve the visual filteringperformance of our nonlinear filter. The quincunx version takes accountof the different correlation structure of the image along differentspatial directions, based on the human vision system (HVS). Iterativeprocessing improves the performance of our algorithm for highlycorrupted images. Numerical simulations show that the quincunx filteringtechnique is extremely robust and efficient, and leads to significantimprovement in different noise-degraded case. The method can be combinedwith any restoration technique to constitute a robust restoration method(for example as the post-processor for wavelet thresholding techniques).We compared the performance of these techniques, both subjectively andquantitatively, with the median filter and two of its recently proposedvariants. Special attention was given to the ability of these methods topreserve the fine image details, such as edges and thin lines. In theexperiments, our filtering gave the best results.

REFERENCES

-   [1] E. Abreu, M. Lightstone, S. K. Mitra, and K. Arakawa, “A new    efficient approach for the removal of impulse noise from highly    corrupted images,” IEEE Trans. Image Processing, vol. 5, pp.    1012-1025, June 1996.-   [2] J. B. Bednar, T. L. Watt, “Alpha-trimmed means and their    relationship to median filter,” IEEE Trans. ASSP, Vol. 32, pp.    145-153, 1984.-   [3] D. R. K. Brownrigg, “The weighted median filter,” Comm. Assoc.    Comput. Mach., Vol. 27, pp. 807-818, 1984.-   [4] H. Hwang, R. A. Haddad, “Adaptive median filters: new algorithms    and results,” IEEE Trans. Image Processing, Vol. 4, No. 4, pp.    499˜502, April 1995.-   [5] B. I. Justusson, “Median filtering: statistics properties,” in    Two-Dimensional Digital Signal Processing, II: Transforms and Median    Filters, Vol. 42, pp. 161-196. New York: Springer Verlag, 1981.-   [6] S. R. Kim, A. Efron, “Adaptive robust impulse noise filtering,”    IEEE Trans. Signal Processing, Vol. 43, No. 8, pp. 1855˜1866, August    1995.-   [7] S. J. Ko, Y. H. Lee, “Center weighted median filters and their    applications to image enhancement,” IEEE Trans. Circuits Syst., Vol.    38, pp. 984-993, Sept. 1991.-   [8] H. M. Lin, A. N. Willson, “Median filters with adaptive length,”    IEEE Trans. Circuits Syst., Vol. 35, pp. 675-690, June 1988.-   [9] A. Scher, F. R. D. Velasco, and A. Rosenfeld, “Some new image    smoothing techniques,” IEEE Trans. Systems, Man, and Cybernetics,    Vol. SMC-10, No. 3, March 1980.-   [10] Z. Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Perceptual    normalized subband image restoration”, IEEE Symposium on    Time-frequency and Time-scale Analysis, N. 144, pp. 469-472,    Pittsburgh, Pa., Oct. 6-9, 1998.-   [11] R. Sucher, “Removal of impulse noise by selective filtering”,    IEEE Proc. Int. Conf. Image Processing, Austin, Tex., November 1994,    pp. 502-506.-   [12] T. Sun, Y. Neuvo, “Detail-preserving median based filters in    image processing,” Pattern Recognition Letter, Vol. 15, pp. 341-347,    April 1994.-   [13] T. D. Tran, R. Safranek, “A locally adaptive perceptual masking    threshold model for image coding,” IEEE Proc. ICASSP, pp. 1882-1885,    1996.-   [14] Z. Wang, D. Zhang, “Restoration of impulse noise corrupted    images using long-range correlation,” IEEE Signal Processing Letter,    Vol. 5, pp. 4-7, Jan. 1998.-   [15] X. You, G. Grebbin, “A robust adaptive estimator for filtering    noise in images,” IEEE Trans. Image Processing, Vol. 4, No. 5, pp.    693˜699, May 1995.

VISUAL MULTIRESOLUTION COLOR IMAGE RESTORATION Introduction

Images are often contaminated by noise. Generally, the possible noisesources of images are photoelectric exchange, photo spots, imperfectionof communication channel transmission, etc. The noise causes speckles,blips, ripples, bumps, ringing and aliasing for visual perception. Thesedistortions not only affect the visual quality of images, but alsodegrade the efficiency of data compression and coding. De-noising andrestoration are extremely important for image processing.

The traditional image processing techniques can be classified into twokinds: linear or non-linear. The principle methods of linear processingare local averaging, low-pass filtering, band-limit filtering ormulti-frame averaging. Local averaging and low-pass filtering onlypreserve the low-band components of the image signal. The original voxelis substituted by an average of its neighboring voxels (within a squarewindow). The mean error may be improved but the averaging process blursthe silhouette and details of the image. Band-limited filters can beutilized to remove the regularly appearing dot matrix, texture and skewlines. But they are useless for irregularly distributed noise.Multi-frame averaging requires the images be still, and the noisedistribution be stationary, which is not the case for motion pictureimages or for a space (time)-varying noisy background. Generally,traditional image processing is always defined on the whole space (time)region, which cannot localize the space (time)-frequency details of asignal. New research evidence shows that non-Guassian and non-stationaryprocesses are important characteristics for the human visual response.

Moreover, traditional image quality is evaluated by the mean squareerror (MSE), which possesses the advantage of a simple mathematicalstructure. For a discrete signal {s(n)} and its approximation {ŝ(n)},n=0, . . . , N−1, the MSE can be defined as $\begin{matrix}{{MSE} = {\frac{1}{N}{\sum\limits_{n = 0}^{N - 1}\quad\lbrack {{\hat{s}(n)} - {s(n)}} \rbrack^{2}}}} & (210)\end{matrix}$However, a MSE based measure, (such as PSNR=log[(255×255)MSE]), does notexactly evaluate image quality that consistent with human perception.The minimum MSE rule may cause serious oscillatory (no monotonic)behavior in the convergence process, which will destroy the originalsmooth transition around the pixels. Commonly used regularizationmethods, such as regularized least squares, may degrade the imagedetails (edges and textures) and result in visual blur.

Human visual perception is more sensitive to image edges which consistof sharp-changes in the neighboring luminance/chrominance scale. Visualperception is essentially adaptive and has variable lenses and focusesfor different visual environments. To protect as much as possible thedetailed information, while simultaneously removing the noise, modemimage processing techniques are predominantly based on non-linearmethods. Commonly, prior to nonlinear filtering, the image edges, aswell as perceptually sensitive textures, are detected and preserved. Thewell-known nonlinear filtering approaches include median filtering andweighted averaging, etc. Median filtering uses the medianluminance/chrominance value of the pixels within the neighboring windowto substitute for the original voxels. This method causes lessdegradation for the slant or square functions, but suppresses the signalimpulses whose widths are shorter than half of the window width. Thus itcan degrade image quality. To protect the edges, weighting averages onlysmooth neighboring pixels with similar luminance/chrominance magnitude.However, a serious shortcoming of these methods is that the width of theweighting-window has to be implemented adaptively. So the large-scale,complicated calculations are required to generate filtering voxels. Ifthe window selected is too wide, more details may be lost.

Human vision system response has attracted much interest for imageprocessing recently [9, 18, 23, 40, 43, 49, 54]. Using some newlydeveloped non-linear methods (such as perceptually optimal stackfiltering [18]) satisfactory restoration images can be obtained.However, this type of nonlinear filter requires a thorough knowledge ofthe ideal image and involves lengthy long-time network training, whichgreatly limits its usefulness.

More efficient, human-vision-system-based image processing techniquespossess the advantages of 1) high de-correlation for convenience ofcompression and filtering; 2) high numerical stability. 3) In addition,in human visual response, the filtering algorithm should enhanceperceptually sensitive information, while simultaneously suppressing thenon-perceptual-sensitive components. 4) Finally, it can be carried outwith real-time processing and is robust.

The space (time)-scale logarithmic response characteristic of thewavelet transform is quite similar to that of the HVS response. Visualperception is sensitive to narrow band, low-frequency components, andinsensitive to wide-band, high frequency components. This feature can bedealt with using the constant-Q analysis of wavelet transforms, whichpossesses fine resolution in the low-band regime, and coarse frequencyresolution in the high band regime. The recently discovered biologicalmechanism of the human visual system shows that both multiorientationand multiresolution are important features of the human visual system.There exist cortical neurons which respond specifically to stimuliwithin certain orientations and frequencies. The visual system has theability to separate signals into different frequency ranges. Theevidence in neurophysiological and psychophysical studies shows thatdirection-selective cortex filtering is much like a 2D-waveletdecomposition representation. The high-pass expansion coefficients of awavelet transform can be regarded as a kind of visible differencepredictor (VDP).

The use of wavelets for the task of image restoration and enhancement isa relatively new but rapidly emerging approach [3, 26, 27, 29, 38, 40].Although there has long been the view that a non-stationary approach mayimprove results substantially compared to a stationary one, the idea ofmultiresolution has not been a prevalent one. Instead, adaptiverestoration techniques have been used to examine problems in the spatialdomain. Various local measures are employed to describe the signalbehavior near a pixel. Because local adaptivity is based explicitly onthe values of the wavelet coefficients, it is very easy to implement andrequires a much smaller effort than the conventional deconvolutionfilter.

To enhance the performance of wavelet techniques for signal-noise-ratioimprovement in signal restoration, it is typical to utilize thresholdingtechniques in multiscale spaces [13, 26, 27, 37, 38]. However, forperceptual image processing dependent on the human vision system (HVS),this technique appears not to be optimal. As is well known, the HVSmodel addresses three main sensitivity variations, namely, the light(luminance/chrominance) level, spatial frequency, and signal content.The visual content of the wavelet coefficients in various subblocks(representing different spatial frequency band) is quite different.Additionally, the contrast level seriously affects the level of athreshold cutoff. Simple multi-channel thresholding methods do not yieldperfect perception quality since they are not fully consistent with HVSresponse. Furthermore, due to the recurrent structure of the subbandfilters, the coefficient strength in each sub-block is also rapidlyvarying. In addition to these difficulties, the problem of color imagerestoration presents a unique problem in that the multiple colorchannels are not orthogonal. Thus, cross-channel correlation must beexploited in order to achieve optimal restoration results. A number ofapproaches have been used to handle the color multichannel imagerestoration problem [2, 3, 31, 49]. However, the re-normalization of thesubband response and the HVS has not yet been studied thoroughly.

The present disclosure deals with these issues and seeks to develop anefficient method for noisy color image restoration. We will concentrateon YCrCb channels because they are relatively de-correlated and can beprocessed independently. Magnitude normalization (MN) is utilized toadjust the wavelet transform coefficients in the different sub-channels.Also, different visual weightings are employed in the threeluminance-chrominance channels (Y, Cr, Cb) to make the wavelet transformcoefficients in these channels better match the vision system responsebetter. These visual weightings have been used before by Waston togenerate a perceptual lossless quantization matrix for image compression[49]. For non-standard image brightness level (contrast), visualsensitivity normalization (VSN) is also needed to fix the cutoffthreshold. We refer to these combined three normalization processes asColor Visual Group Normalization (CVGN).

In applications, recently developed interpolating Lagrange wavelets [41,50] are utilized for color image decomposition and reconstruction. Thiskind of interpolating wavelet displays a slightly better smoothness andmore rapidly time-frequency decay than commonly used wavelets. Moreover,the interpolating processing enables us to utilize a parallelcalculation structure for efficient real-time implementation. Mostimportantly, our multiresolution image processing method is extremelyrobust in not requiring prior knowledge of either an ideal image ornoise level.

COLOR MODULATION

Commonly used color models are of three kinds:

(1) Computer Graphics Color Space

-   CMY: Cyan, Magenta and Yellow-   HLS: Hue, Lightness and Saturation-   HSV (HSB): Hue, Saturation and Value (Brightness)-   RGB: Red, Green and Blue    (2) TV Broadcast Signal Color Space-   YCrCb: Intensity, Color-red and Color-blue-   YIQ: Luminance, In-Phase and Quadrature    (3) The CIE (Commission Internationale de Eclairage) Uniform Color    Space-   CIEXYZ: Standard primaries X, Y and Z-   CIELab: Luminance, a value and b value-   CIELuv: Luminance, u value and v value-   CIExyY: x-y coordinate and Luminance

A color vector is usually represented by its three components C=[R, G,B] of red, green and blue primary signals (FIG. 66), with each signalbeing represented with 8-bit precision (i.e. an integer range of[0,255]). In color cathode ray tube (CRT) monitors and raster graphicsdevices, the primarily used color model is red, green, and blue (RGB).It is also by far the most commonly used model for computer monitors.The Cartesian coordinate system is employed in this model. The RGBprimaries are additive, such that individual contributions of eachprimary are added for the creation of a new color. The model is based onthe tri-stimulus theory of vision and is a hardware-oriented model.

This model can be represented by the unit cube which is defined on R, G,and B axes (FIG. 67). The line that runs from Black (0,0,0) to White(1,1,1) is the gray scale line [56]. The values of R, G and B should bethe same (R=G=B) in order to have an achromatic (colorless) pixel. Afull view of a colored RGB cube can be seen here. The RGB color model isadditive, so that:Red+Green=YellowRed+Blue=MagentaGreen+Blue=CyanRed+Green+Blue=White   (211)

The color cube can also be projected as a hexagon as FIG. 68, with thelightest point in the middle. This alternative representation is usefulfor understanding the relationship to other proposed color models. Forexample, it helps to visualize the color transformations between severalother color models (e.g. CMY, HSV, HLS, etc.). With the use of ahexagon, a color RGB cube can be represented [56].

Let RL, GL, and BL, be values of three color primaries that are equal to(or proportional to) the measured luminance of the color primaries. Itshould be noted that these primaries cannot be displayed directlybecause most monitors exhibit a nonlinear relationship between the inputvalue of the color signal and the corresponding output luminance. Infact, the nonlinearity for a particularly primary can be approximated bya power law relationship:C _(L) 32 C ^(γ) , C=[R, G, B]  (212)

where C is the input value of the primary, C_(L) is the luminance of theprimary, and γ is a value that usually falls between 2 and 3, dependingon the monitor used. The nonlinearity can also be characterized by morecomplicated models. In order to account for this nonlinearity, atransform on the linear color primaries of the form

 R=R _(L) ^(1/γ) , G=G _(L) ^(1/γ) , B=B _(L) ^(1/γ)  (213)

is typically performed as display on the given monitor. Thegamma-corrected coordinates are also used to characterize manydevice-independent color spaces. One such color space is SMPTE RGB colorspace, which has been selected as a television standard. The value of γfor this color space is 2.2. All the color images used in this work aretaken to be in SMPTE gamma-corrected RGB coordinates.

Since the human vision system (HVS) perceives a color stimulus in termsof luminance and chrominance attributes, rather than in terms of R, G, Bvalues, we propose to transform the image to a luminance-chrominancespace prior to performing the quantization. The color representationscheme used in this work is the CCIR-601-1 eight-bit digital codingstandard, comprising a luminance (Y) and chrominance (Cb/Cr) components,with black at luma code 16 and white at luma code 235 (instead of thefull 8-bit range of 0 to 255). To this end, we pick the YCrCb componentcolor space that is related to the SMPTE RGB space by a simple lineartransformation. Assuming that R, G, B occupy the range 0-255, thetransform is given byY=0.298993R+0.587016G+0.113991BCr=128+0.713267(R−Y)Cb=128+0.564334(B−Y),   (214)where the Y, Cr, and Cb values have been scaled to the range 0-255.Hence forth, 3D color vectors will be assumed to be in YCrCbcoordinates. Since YCrCb is a linear transform of a gamma-corrected RGBspace, it is also a linear gamma-corrected space: Y is thegamma-corrected luminance component representing achromatic colors, theCr coordinate describes the red-green variation of the color, and the Cbcoordinate describes the yellow-blue variation of the color. Theconversion to YCrCb is one of the key factors enabling us to achievehigh image quality.

Digital grayscale images typically contain values that representso-called gamma-corrected luminance Y. Here Y is a power function ofluminance, with an exponent of around 1/2.3. Three gamma-correctednumbers, red, green, and blue components (RGB), are used to representeach pixel in the case of color images. In the case of a gray image,each pixel is represented by a single brightness. In this treatment,each pixel is transformed from the original color representation (forexample RGB) to a color representation that consists of one brightnesssignal and two color signals (such as the well-known TV broadcast colorspace YCbCr).

TABLE 6 Display resolution comparison Resolution Distance DVR Display(pixels/inch) (inches) (pixels/degree) Computer Display 72 12 15.1 LowQuality Printing 300 12 62.8 High Quality Printing 1200 12 251.4 HDTV 4872 60.3

VISUAL SENSITIVITY OF WAVELET COEFFICIENTS

The visibility of wavelet transform coefficients will depend upon thedisplay visual resolution [49] in pixel/degree. Given a viewing distanceV in inches and a display resolution d in pixel/inch, the effectivedisplay visual resolution (DVR) R in pixel/degree of visual angle isR=dVtan(π/180)≈dV/57.3   (215)A useful measurement is that the visual resolution is the viewingdistance in pixels (dV) divided by 57.3. Table 1 provides someillustrative examples. For example, the HDTV assumes 1152 active linesat a viewing distance of 3 picture heights [49].

On each decomposition level, wavelet coefficients are divided into foursubblocks, LL, HL, LH, and HH. The detailed subblocks HL, LH and HHrepresent three different decomposition orientations, horizontal,vertical, and diagonal, respectively. As the layer increases, thebandwidth of the equivalent subband filters decreases by a factor oftwo, and thus the frequency resolution doubles. Correspondingly, thespace (time) resolution (display resolution) decreases by a factor oftwo. A subblock of wavelet coefficients corresponds to a spatialfrequency band. For a display resolution of R pixel/degree, the spatialfrequency f of level j isƒ=2^(−j) R   (216)

As discussed in section I, the display gamma is 2.3. The R, G, B truecolor image is modulated into Y, Cr, Cb channels. The model thresholdsfor three-color channels are different from each other. For example, thejust-noticeable quantization threshold of Y is generally about a factorof two below that of Cr, which is in turn about a factor of two belowthe Cb curve at each spatial frequency. Note that this differenceusually declines at higher spatial frequency [49]. The Cb curve issomewhat broader than Y or Cr. This broadening is likely due to theintrusion of a luminance-detecting channel at high frequencies and highcontrasts. Because the Cb color axis is not orthogonal to the humanluminance axis, the Cb colors do have a luminance component.

The contrast sensitivity declines when the spatial frequency increases,whereas, the size of stimuli decreases. This mode is adapted toconstruct the “perceptual lossless” response magnitude for normalizingin subblock (j,m) according to the visual response.

VISUAL GROUP NORMALIZATION

The main objective of wavelet signal filtering is to preserve importantsignal components, and efficiently reduce noisy components. Forperceptual images, it is most important to protect the signal components(always represented by the luminance/chrominance levels) which aresensitive to human eyes. To achieve this goal, we utilize the magnitudesof the filter response and the human vision response.

Magnitude Normalization

Wavelet coefficients can be regarded as the output of the signal passingthrough equivalent decomposition filters (EDF). The responses of the EDFare the combination of several recurrent subband filters at differentstages [19, 28, 34, 37, 38, 40, 41]. The EDF amplitudes of varioussub-blocks differ greatly. Thus, the magnitude of the decompositioncoefficients in each of the sub-blocks cannot exactly reproduce theactual strength of the signal components. To adjust the magnitude of theresponse in each block, the decomposition coefficients are re-scaledwith respect to a common magnitude standard. Thus EDF coefficients,C_(m)(k), in block m should be multiplied with a magnitude scalingfactor, λ_(m), to obtain an adjusted magnitude representation. We choosethis factor to be the reciprocal of the maximum magnitude of thefrequency response of the equivalent wavelet decomposition filter onnode (j,m) $\begin{matrix}{{\lambda_{j,m} = \frac{1}{\sup\limits_{\omega \in \Omega}\{ {{{LC}_{j,m}(\omega)}} \}}},\quad{\Omega = \lbrack {0,{2\quad\pi}} \rbrack}} & (217)\end{matrix}$This idea was recently extended to Group Normalization (GN) of waveletpackets for signal processing [37, 38, 40, 41] and was presented to leadthe optimal performance.Perceptual Lossless Normalization

Because an image can be regarded as a signal source based on the humanvisual system, using a just-noticeable distortion profile we canefficiently remove the visual redundancy from decomposition coefficientsand normalize them with respect to the standard of perceptualimportance. A mathematical model for perception efficiency, based on theamplitude nonlinearity in different frequency bands, has been presentedby Watson, et al. [49], which can be used to construct the “perceptuallossless” response magnitude Y_(j,m,ν) for normalizing visual responsein different luminance/chrominance spaces. We extend the definition toluminance/chrominance mode according to $\begin{matrix}{Y_{j,m,v} = {a_{v}\quad 10^{{k_{v}{({\log\quad\frac{2^{j}f_{0,v}d_{m,v}}{R_{v}}})}}^{2}}}} & (218)\end{matrix}$where a_(ν) defines the minimum threshold, k_(ν) is a constant, R_(ν) isthe Display Visual Resolution (DVR),ƒ_(0,ν) is the spatial frequency,and d_(m,ν) is the directional response factor, in eachluminance/chrominance channel ν.

The parameters d_(LL,ν), and d_(HH,ν), represent the thresholds fororientations LL and HL as frequency shifts relative to threshold fororientations LH and HL. From the nature of dyadic wavelets, theorientation LL possesses a spectrum that is approximately a factor oftwo lower in spatial frequency than orientation LH or HL. This wouldsuggest a factor of d_(LL,ν)=2. However, in a vision system responsemechanism, at orientation LL the signal energy is spread over allorientations, which implies less visual efficiency than when the energyis concentrated over a narrow range, as is the case for the spectra oforientation HL or LH. Thus, the threshold should be increased, which canbe realized by a slight reduction in ƒ₀. The final value for d_(LL,ν) istherefore less than 2.

For orientation HH, similar effects exist. First, the Cartesiansplitting of the spectrum makes the spatial frequency of orientation HHabout √{square root over (2)} higher than that of the orientations HL,and LH. Thus d_(HH,ν) can be taken as d_(HH,ν)=2^(−½). As we mentionedbefore, the spectrum along orientation HH is spread over two orthogonalorientations (45° and 135° ), which should result in a log thresholdincrease of about 2^(1/4) (a shift of 2^(−¼)) or a total prediction ofd_(HH,ν)=2^(−¾)=0.59. Finally, the well-known oblique effect [49], willcause a final small amount of threshold elevation.

The chromatic channels Cb and Cr are usually each down-sampled by afactor of two in both horizontal and vertical directions, because thesensitivity of human vision to chromatic variation is weaker than it isto luminance variation. In the calculation of the lossless quantizationmatrix, we should in this case make some correction. For example, if thedisplay visual resolution is known, as the chroma that is down-sampledby two in each dimension, the corrected value will be half of that. Thiscorrected value is used to adjust the perceptual lossless normalizationin Cr and Cb channels.

Visual Sensitivity Normalization

Visual sensitivity in each luminance/chrominance channel ν is defined asthe inverse of the contrast required to produce a threshold response[9],S _(ν)=1/C _(ν)  (219)where C_(ν) is generally referred to simply as the threshold. TheMichelson definition of contrast,C _(ν)=(L _(max,ν) −L _(mean,ν))/L _(mean,ν)  (220)is used, where L_(max,ν) and L_(mean,ν) refer to the maximum and meanluminances of the waveform in luminance/chrominance channel ν.Sensitivity can be thought of as a gain, although various nonlinearitiesof the visual system require caution in the use of this analogy. Thevariations in sensitivity as a function of light level are primarily dueto the light-adaptive properties of the retina and are referred to asthe amplitude nonlinearity of the HVS. The variations as a function ofspatial frequency are due to the optics of the eye combined with theneural circuitry; these combined effects are referred to as the contrastsensitivity function (CSF). Finally, the variations in sensitivity as afunction of signal content referred to as masking, are due to thepost-receptoral neural circuitry.

Combining the perceptual lossless normalization, the visual sensitivitynormalization and the magnitude normalized factor λ_(j,m), we obtain theperceptual lossless quantization matrix Q_(j,m,ν)Q _(j,m,ν)=2C _(ν) Y _(j,m,ν) λ _(j,m)   (221)This treatment provides a simple human-vision-based threshold techniquefor the restoration of the most important perceptual information in animage. We call the combination of the above-mentioned threenormalizations Color Visual Group Normalization (CVGN) of wavelettransform coefficients.

RESULTS OF EXAMPLE APPLICATIONS

Our study suggests that the analyzing functions presented in thisdisclosure can improve the visualization of features of importance tocolor images. As discussed earlier, the color channel application of theColor Visual Group Normalization (VGN) technique is designed to improvethe performance of our interpolating Lagrange wavelet transform [41] inimage processing. Utilizing Visual Group Normalization, the rawmagnitudes of the transform coefficients in different luminance andchrominance channels can be normalized to represent exactly the visualperceptual strength of signal components in each subband. Moreover, theefficient non-linear filtering method—softer logic masking (SLM)technique [38], provides robust edge-preservation for image restoration,and removes the haziness encountered with the commonly used hard-logicfiltering techniques.

In our study, the original benchmark color photo of Lena was cropped toa matrix size of 512×512. Another color picture is digitized by Kodakdigital camera with matrix size 512×768. The original images possessclear edges, strong contrast and brightness. FIG. 69(a) is the typicalnoisy Lena image degraded by adding Gaussian random noise. A simplelow-pass filter smoothes out the noise but also degrades the imageresolution, while a simple high-pass filter can enhance the textureedges but will also cause additional distortion. We choose 2D half-bandLagrange wavelets as the multiresolution analysis tools for imageprocessing.

The median filtering (with a 3×3 window) result of Lena is shown in FIG.69(b), which is edge-blurred with low visual quality. The speckled noisehas been changed into bumps. It is evident that our Color Visual GroupNormalization technique yields better contrast and edge-preservationresults and provides a more natural color description of this image(FIG. 69(c)).

For a second example, the noisy picture, result of median filtering andour result are shown in FIG. 70(a), 70(b) and 70(c), respectively. Ourprevious conclusion pertain, showing that the CVGN technique is alsosuitable for image processing of pictures having a differentheight-width ratio.

CONCLUSIONS

Blind image restoration and de-noising are very difficult tasks. Becauseof the complex space-frequency distribution and statistical feature ofimages, there is almost no definite feature discrimination between imageand noise background. Even if prior knowledge of the noise is available,a perfect extraction is theoretically impossible. The key problem insignal processing is how to restore images “naturally” from a noisybackground or from an encoded quantization space. For a de-noisingscheme based on statistical properties, an adaptive signal-dependenttransform is needed to generate a concentrated representation of signalcomponents. This is based on the assumption that noise or cluttercomponents in the signal subspace are relatively scattered with lowermagnitudes.

The logrithmic frequency band distribution of our wavelet decompositionis matched well with the characteristics of human vision responses toobtain compact signal representation. Both multiorientation andmultiresolution are known features of the human visual system. Thereexist cortical neurons that respond specifically to stimuli withincertain orientations and frequencies. The 2-D wavelet decompositionmimics the cortex filtering of the HVS. Visual group normalization isproposed to normalize the frequency response so that the waveletcoefficients can exactly represent the perceptual strength of signalcomponents. The color channel based VGN technique results in athresholding method that is efficient for visual feature extraction.

We also employ a modified version of a well-known threshold method(termed the Perceptual Softer Logic Masking (PSLM) technique [38, 40])for image restoration, which dealing with extremely noisy backgrounds.This technique better preserves the important visual edges and contrasttransition portions of an image than does the traditional method, and itis readily adaptable to human vision. The smooth transition around thecut-off region of the filter can efficiently remove the Gibbs'oscillation in the signal reconstruction, and decrease the appearance ofringing and aliasing. A symmetric extension decomposition [6] isutilized to remove non-continuous boundary effects. The interpolationalgorithm made possible by use of an interpolating wavelet transform[12, 14, 15, 16, 25, 35, 36, 41] enables us to utilize a parallelcomputational strategy that is convenient for real-time implementation.

To some extent, color visual group normalization provides a measurementfor handling raw filter coefficients in a wavelet transform. Thehierarchical filtering of Softer-logic Masking can protect more signalcomponents than a simple single layer nonlinear method. The concept ofVisual Lossless Quantization (VLQ) presented in [49] can lead to apotential breakthrough compared to the traditional Shannonrate-distortion theory in information processing. We have compared ourapproach with the commonly used median filtering method for de-noising.The results show that the proposed method is robust and provides thebest quality for color image filtering of which we are aware.

REFERENCES

-   [1] R. Ansari, C. Guillemot, and J. F. Kaiser, “Wavelet construction    using Lagrange halfband filters,” IEEE Trans. CAS, vol. 38, no. 9,    pp. 1116-1118, 1991.-   [2] R. Balasubramanian, C. A. Bouman, and J. P. Allebach,    “Sequential scalar quantization of color images,” J. Electronic    Imaging, vol. 3, no. 1, pp. 45-59, January 1994-   [3] M. R. Banham, A. K. Katsaggelos, “Digital image restoration,”    IEEE SP Mag., pp. 24-41, March 1997-   [4] R. Baraniuk, D. Jones, “Signal-dependent time-frequency analysis    using a radially Gaussian kernel,” Signal Processing, Vol. 32, pp.    263-284, 1993.-   [5] A. C. Bovic, T. S. Huang, and D. C. Munson, “A generalization of    median filtering using linear combinations of order statistics,”    IEEE Trans. ASSP, vol. 37, pp. 2037-2066, 1989-   [6] C. M. Brislawn, “Preservation of subband symmetry in multirate    signal coding,” IEEE Trans. SP, vol. 43, no. 12, pp. 3046-3050,    1995.-   [7] C. K. Chui, An Introduction to Wavelets, Academic Press, New    York, 1992.-   [8] C. K. Chui, Wavelets: A Tutorial in Wavelet Theory and    Applications, Academic Press, New York, 1992.-   [9] S. Daly, “The visible difference predictor: an algorithm for the    assessment of image fidelity,” in Digital Images and Human Vision,    pp. 179-206, ed. By A. B. Waston, MIT Press, 1993-   [10] I. Daubechies, “Orthonormal bases of compactly supported    wavelets”, Comm. Pure and Appl. Math., vol. 41, no. 11, pp. 909-996,    1988.-   [11] I. Daubechies, “The wavelet transform, time-frequency    localization and signal analysis,” IEEE Trans. Inform. Theory, Vol.    36, No. 5, pp. 961-1003, September 1990.-   [12] G. Deslauriers, S. Dubuc, “Symmetric iterative interpolation    processes,” Constructive Approximations, vol. 5, pp. 49-68, 1989.-   [13] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.    Information Theory, vol. 41, no. 3, pp. 613-627, 1995.-   [14] D. L. Donoho, “Interpolating wavelet transform,” Preprint,    Stanford Univ., 1992.-   [15] S. Dubuc, “Interpolation through an iterative scheme”, J. Math.    Anal. and Appl., vol. 114, pp. 185˜204, 1986.-   [16] A. Harten, “Multiresolution representation of data: a general    framework,” SIAM J. Numer. Anal., vol. 33, no. 3, pp. 1205-1256,    1996.-   [17] J. J. Huang, E. J. Coyle, G. B. Adams, “The effect of changing    the weighting of errors in the mean absolute error criterion upon    the performance of stack filters,” Proc. 1995 IEEE Workshop on    Nonlinear Signal and Image Processing, 1995.-   [18] J. J. Huang, E. J. Coyle, “Perceptually Optimal Restoration of    Images with Stack Filters,” Proc. 1997 IEEE/EURASIP Workshop on    Nonlinear Signal and Image Processing, Mackinac Island, Mich., 1997-   [19] C. Herley, M. Vetterli, “Orthogonal time-varying filter banks    and wavelet packets,” IEEE Trans. SP, Vol. 42, No. 10, pp.    2650-2663, October 1994.-   [20] C. Herley, Z. Xiong, K. Ramchandran and M. T. Orchard, “Joint    Space-frequency Segmentation Using Balanced Wavelet Packets Trees    for Least-cost Image Representation,” IEEE Trans. Image Processing,    vol. 6, pp. 1213-1230, September 1997.-   [21] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri,    “Analytic banded approximation for the discretized free    propagator,” J. Physical Chemistry, vol. 95, no. 21, pp. 8299-8305,    1991.-   [22] L. C. Jain, N. M. Blachman, and P. M. Chapell, “Interference    suppression by biased nonlinearities,” IEEE Trans. IT, vol. 41, no.    2, pp. 496-507, 1995.-   [23] N. Jayant, J. Johnston, and R. Safranek, “Signal compression    based on models of human perception”, Proc. IEEE, vol. 81, no. 10,    pp. 1385˜1422, 1993.-   [24] J. Kovacevic, and M. Vetterli, “Perfect reconstruction filter    banks with rational sampling factors,” IEEE Trans. SP, Vol. 41, No.    6, pp. 2047-2066, June 1993.-   [25] J. Kovacevic, W. Swelden, “Wavelet families of increasing order    in arbitrary dimensions,” Submitted to IEEE Trans. Image Processing,    1997.-   [26] A. F. Laine, S. Schuler, J. Fan and W. Huda, “Mammographic    feature enhancement by multiscale analysis,” IEEE Trans. MI, vol.    13, pp. 725-740, 1994.-   [27] S. Liu and E. J. Delp, “Multiresolution Detection of Stellate    Lesions in Mammograms,” Proc. 1997 IEEE International Conference on    Image Processing, October 1997, Santa Barbara, pp. II 109-112.-   [28] S. Mallat, “A theory for multiresolution signal decomposition:    the wavelet representation,” IEEE Trans. PAMI, Vol. 11, No. 7, pp.    674-693, July 1989.-   [29] S. Mallat, S. Zhong, “Characterization of signals from    multiscale edges,” IEEE Trans. PAMI, Vol. 14, No. 7, pp. 710-732-   [30] Y. Meyer, Wavelets Algorithms and Applications, SIAM Publ.,    Philadelphia 1993.-   [31] M. Orchard, C. Bouman, “Color quantization of images,” IEEE    Trans. SP, vol. 39, no. 12, pp. 2677-2690, December 1991.-   [32] K. Ramchandran, M. Vetterli, “Best wavelet packet bases in a    rate-distortion sense,” IEEE Trans. Image Processing, Vol. 2, No. 2,    pp. 160-175, April 1993.-   [33] K. Ramchandran, Z. Xiong, K. Asai and M. Vetterli, “Adaptive    Transforms for Image Coding Using Spatially-varying Wavelet    Packets,” IEEE Trans. Image Processing, vol. 5, pp. 1197-1204, July    1996.-   [34] O. Rioul, M. Vetterli, “Wavelet and signal processing,” IEEE    Signal Processing Mag., pp. 14-38, October 1991.-   [35] N. Saito, G. Beylkin, “Multiscale representations using the    auto-correlation functions of compactly supported wavelets,” IEEE    Trans. Signal Processing, Vol. 41, no. 12, pp. 3584-3590, 1993.-   [36] M. J. Shensa, “The discrete wavelet transform: wedding the a    trous and Mallat algorithms”, IEEE Trans. SP, vol. 40, no. 10, pp.    2464˜2482, 1992.-   [37] Z. Shi, Z. Bao, “Group-normalized processing of complex wavelet    packets,” Science in China (Serial E), Vol. 26, No. 12, 1996.-   [38] Z. Shi, Z. Bao, “Group-normalized wavelet packet signal    processing”, Wavelet Application IV, SPIE Proc., vol. 3078, pp.    226˜239, 1997.-   [39] Z. Shi, Z. Bao, “Fast image coding of interval interpolating    wavelets,” Wavelet Application IV, SPIE Proc., vol. 3078, pp.    240-253, 1997.-   [40] Z. Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Perceptual    image processing using Gauss-Lagrange distributed approximating    functional wavelets,” submitted to IEEE SP Letter, 1998.-   [41] Z. Shi, G. W. Wei, D. J. Kouri, D. K. Hoffman, and Z. Bao,    “Lagrange wavelets for signal processing,” submitted to IEEE Trans.    Image Processing, 1998.-   [42] W. Swelden, “The lifting scheme: a custom-design construction    of biorthogonal wavelets,” Appl. And Comput. Harmonic Anal., vol. 3,    no. 2, pp. 186˜200, 1996.-   [43] T. D. Tran, R. Safranek, “A locally adaptive perceptual masking    threshold model for image coding,” Proc. ICASSP, 1996.-   [44] M. Unser, A. Adroubi, and M. Eden, “The L₂ polynomial spline    pyramid,” IEEE Trans. PAMI, vol. 15, no. 4, pp. 364-379, 1993.-   [45] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part I: system-theoretic fundamentals,” IEEE    Trans. SP, Vol. 43, No. 5, pp. 1090-1102, May 1995.-   [46] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part II: the FIR case, factorizations, and    biorthogonal lapped transforms,” IEEE Trans. SP, Vol. 43, No. 5, pp.    1103-1115, May 1995.-   [47] M. Vetterli, C. Herley, “Wavelet and filter banks: theory and    design,” IEEE Trans. SP, Vol. 40, No. 9, pp. 2207-2232, September    1992.-   [48] J. D. Villasenor, B. Belzer, and J. Liao, “Wavelet filter    evaluation for image processing,” IEEE Trans. IP, vol. 4, no. 8, pp    1053-1060, 1995.-   [49] A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor,    “Visibility of wavelet quantization noise,” IEEE Trans. Image    Processing, vol. 6, pp. 1164-1175, 1997.-   [50] G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Wavelets and    distributed approximating functionals,” Computer Phys. Comm., in    press.-   [51] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,    “Lagrange distributed approximating Functionals,” Physical Review    Letters, Vol. 79, No. 5, pp. 775˜779, 1997.-   [52] Z. Xiong, K. Ramchandran and M. T. Orchard, “Space-frequency    Quantization for Wavelet Image Coding,” IEEE Trans. Image    Processing, vol. 6, pp. 677-693, May 1997.-   [53] J. Yoo, K. L. Fong, J. J. Huang, E. J. Coyle, and G. B. Adams,    “Fast Algorithms for Designing Stack Filters,” submitted to IEEE    Trans. on Image Processing.-   [54] C. Zetzsche, E. Barth, and B. Wegmann, “The importance of    intrinsically two-dimensional image features in biological vision    and picture coding,” in Digital Images and Human Vision, pp.    109-138, ed. By A. B. Waston, MIT Press, 1993-   [55] S. H. Zhang, Z. Bao, etc. “Target extraction from strong    clutter background,” Tech. Rep., National Key Lab. of Radar Signal    Processing, Xidian University, 1994

MAMMOGRAM ENHANCEMENT USING GENERALIZED SINC WAVELETS Introduction

Interpolating Distributed Approximating Functionals (DAFs), designed asa set of envelop-modulated interpolants, are generated byGaussian-modulated Sinc, Hermit, or Lagrange functionals [15, 31, 40,41]. Such DAFs are smooth and decaying in both time and frequencyrepresentations and have been used for numerically solving variouslinear and nonlinear partial differential equations with extremely highaccuracy and computational efficiency. Examples include DAF-simulationsof 3-D reactive quantum scattering, the Kuramoto-Sivashinsky equationsdescribing flow pattern dynamics for a circular domain, the sine-Gordonequation near homoclinic orbits, and a 2-D Navier-Stokes equation withnon-periodic boundary conditions. Because the interpolating core shellof the fundamental DAF is the interpolating Sinc functional (the ideallow-pass filter), it is naturally useful to construct interpolatingwavelets from these DAFs for use in signal processing.

The theory of interpolating wavelets has attracted much attentionrecently [1, 8, 10, 11, 12, 19, 20, 26, 27, 30, 31, and 32]. Itpossesses the attractive characteristic that the wavelet coefficientsare obtained from linear combinations of discrete samples rather thanfrom traditional inner product integrals. Mathematically, variousinterpolating wavelets can be formulated in a biorthogonal setting andcan be regarded as an extension of the auto-correlation shell waveletanalysis [26], and halfband filters [1]. Harten has described a kind ofpiecewise polynomial method for biorthogonal interpolating waveletconstruction [12]. Swelden independently develops this method as thewell-known interpolating “lifting scheme” theory [32], which can beregarded as a special case of the Neville filters [19]. Unlike theprevious method for constructing the biorthogonal wavelets, whichattempts to explicitly solve the relevant coupled algebraic equations[10], the lifting scheme enables one to construct a custom-designedbiorthogonal wavelet transform just assuming a single low-pass filter(such as DAF filter or scaling function) without iterations.

Generally speaking, the lifting-interpolating wavelet theory is closelyrelated to the finite element technique in the numerical solution ofpartial differential equations, the subdivision scheme for interpolationand approximation, multi-grid generation and surface fitting techniques.The most attractive feature of the approach is that discrete samplingsare made identical to wavelet multiresolution analysis. Without anypre-conditioning or post-conditioning processes for accurate waveletanalysis, the interpolating wavelet coefficients can be implementedusing a parallel computational scheme.

In this paper, the Sinc-DAF is employed to construct new biorthogonalinterpolating wavelets—DAF wavelets and associated DAF-filtersspecifically for use in mammogram decomposition. Two kinds of differentbiorthogonal interpolating DAF wavelets (B-spline-enveloped Sincwavelets and Gaussian-enveloped DAF wavelets), as the examples of thegeneralized interpolating Sinc wavelets, have been discussed thoroughly.Because the finite length cutoff implementation of the Sinc (ideallow-pass) filter causes the Gibbs oscillations, the key idea for the DAFwavelet construction is to introduce bell-shaped weighted windows toimprove the characteristics of localization in time-frequency plane, andto ensure perfectly smooth and rapid decay.

To obtain excellent reconstruction quality of a digital mammogram, humanvisual sensitivity is utilized to construct the visual groupnormalization (VGN) technique, which is used to re-scale the waveletdecomposition coefficients for perceptual adapted reconstructionaccording to human perception. Softer Logic Masking (SLM) is an adjustedde-noising technique [29], derived to improve the filtering performanceof Donoho's Soft Threshold method [9]. The SLM technique moreefficiently preserves important information and edge transition in amanner particularly suited to human visual perception. A nonlinearcontrast stretch and enhancement functional is easily realized forwavelet multiscale gradient transformation and feature-sensitive imagereconstruction, which enables us to obtain accurate space-localizationof the important features of the mammogram. The combined techniques cangreatly improve the visualization of low-contrast components of amammogram, which is important for diagnosis. Additionally, interpolatingDAF-wavelet image processing can be implemented as an interpolationmethod. The coefficient calculations therefore only involve simpleadd/multiply operations. This is extremely efficient for fastimplementation.

As an example application of Sinc-DAF wavelets, we select mammogramimage processing, de-noising and enhancement, because of its huge datasize, complicated space-frequency distribution and complex perceptualdependent characteristics. Perceptual signal processing has thepotential of overcoming the limits of the traditional ShannonRate-Distortion (R-D) theory for perception-dependent information, suchas images and acoustic signals. Previously, Ramchandran, Vetterli,Xiong, Herley, Asai, and Orchard have utilized a rate-distortioncompromise for image compression implementation [14, 23, 24, and 42].Our recently derived Visual Group Normalization (VGN) technique [31] canlikely be used with the rate-distortion compromise to generate aso-called Visual Rate-Distortion (VR-D) theory to improve imageprocessing further.

GENERALIZED SINC WAVELETS

Sinc Wavelets

The π band-limited Sinc function,φ(x)=sin(πx)/(πx)C ^(∞)  (222)in Paley-Wiener space, constructs the interpolating functionals. Everyband-limited function ƒ∈L²(R) can be reconstructed by the equation$\begin{matrix}{{f(x)} = {\sum\limits_{k}\quad{{f(k)}\quad\frac{\sin\quad{\pi( {x - k} )}}{\pi( {x - k} )}}}} & (223)\end{matrix}$where the related wavelet function—Sinclet is defined as (see FIG. 71)$\begin{matrix}{{\psi\quad(x)} = \frac{{\sin\quad{\pi( {{2x} - 1} )}} - {\sin\quad{\pi( {x - {1/2}} )}}}{\pi( {x - {1/2}} )}} & (224)\end{matrix}$

The scaling Sinc function is the well-known ideal low-pass filter whichpossesses the filter response $\begin{matrix}{{H(\omega)} = \{ \begin{matrix}{1,} & {{\omega } \leq {\pi/2}} \\{0,} & {{\pi/2} < {\omega } \leq \pi}\end{matrix} } & (225)\end{matrix}$

Its impulse response can be described as $\quad\begin{matrix}{{h\lbrack k\rbrack} = {{\frac{1}{2\quad\pi}{\int_{{- \pi}/2}^{\pi/2}{{\mathbb{e}}^{j\quad k\quad\omega}\quad{\mathbb{d}\omega}}}} = \frac{\sin\quad( {\pi\quad{k/2}} )}{( {\pi\quad k} )}}} & (226)\end{matrix}$

The so-called half-band filter only possesses non-zero impulses at oddinteger samples, h(2k+1), while at even integers, h[2k]=0, except fork=0.

However, this ideal low-pass filter can not be implemented physically.Because the digital filter is an IIR (infinite impulse response)solution, its digital cutoff FIR (finite impulse response)implementation will introduce Gibbs phenomenon (overshot effect) inFourier space, which degrades the frequency resolution as shown in FIG.72.

The explicit compactly supported Sinc scaling function and wavelet, aswell as their biorthogonal dual scaling function and wavelet, are shownin FIG. 73. We find that the cutoff Sinc has decreased regularity, whichis manifested by a fractal-like behavior, which implies poor timelocalization.

B-Spline Sinc Wavelets

Because the ideal low-pass Sinc wavelet can not be implemented “ideally”by FIR (finite impulse response) filters, a windowed weighting techniqueis introduced here to eliminate the cutoff singularity, and improve thetime-frequency localization of the Sinc wavelets.

First, we define a symmetric Sinc interpolating functional shell as$\begin{matrix}{{P(x)} = \frac{\sin( {\pi\quad{x/2}} )}{\pi\quad x}} & (227)\end{matrix}$

Multiplying by a smooth window, which vanishes gradually at the exactzeros of the Sinc functional, will lead to more regular interpolatingwavelets and equivalent subband filters (as shown in FIGS. 74 and 75).

For example, we select a well-defined B-spline function as the weightwindow. Then the scaling function (mother wavelet) can be defined as aninterpolating B-spline Sinc functional (BSF) $\begin{matrix}{{\phi_{M}(x)} = {{\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}\quad{P(x)}} = {\frac{\beta^{N}( {x/\eta} )}{\beta^{N}(0)}\quad\frac{\sin\quad( {\pi\quad{x/2}} )}{\pi\quad x}}}} & (228)\end{matrix}$where N is the B-spline order, η is the scaling factor to control thewindow width. To ensure coincidence of the zeroes of the B-spline andthe Sinc shell, we set2M+1=ηx(N+1)/2   (229)To ensure the interpolation condition, h(2k)=0, k≠0 , it is easy to showthat when the B-spline order N=4k+1, η can be any odd integer (2k+1). IfN is even integer, then η can only be 2. When N=4k−1, we can notconstruct the interpolating shell using the definition above. Theadmission condition can be expressed as $\begin{matrix}\{ \begin{matrix}{{\eta = 2},} & {N = {2i}} \\{{\eta = {{2k} + 1}},} & {N = {{4i} + 1}}\end{matrix}  & (230)\end{matrix}$From the interpolation relation $\begin{matrix}{{\phi(k)} = \{ {{\begin{matrix}{1,\quad{k = 0}} \\{0,\quad{k \neq 0}}\end{matrix}\quad k} \in Z} } & (231)\end{matrix}$and the self-induced two-scale relation $\begin{matrix}{{\phi(x)} = {\sum\limits_{k}{{\phi( {k/2} )}\quad\phi\quad( {{2x} - k} )}}} & (232)\end{matrix}$it is easy to verify thath(k)=φ_(M)(k/2)/2, k=−2M+1, 2M−1   (233)

GAUSSIAN-SINC DAF WAVELETS

We can also select a class of distributed approximating functionals,e.g., the Gaussian-Sinc DAF (GSDAF) as our basic scaling function toconstruct interpolating scalings, $\begin{matrix}{{\phi_{M}(x)} = {{{W_{\sigma}(x)}\quad{P(x)}} = {{W_{\sigma}(x)}\quad\frac{\sin( {\pi\quad{x/2}} )}{\pi\quad x}}}} & (234)\end{matrix}$where W_(σ)(x) is a window function which is selected as a Gaussian,W _(σ)(x)=e ^(−x) ² ^(/2σ) ²   (235)Because it satisfies the minimum frame bound condition in quantumphysics, it will improve the time-frequency resolution of theWindowed-Sinc wavelet. Here σ is a window width parameter, and P(x) isthe Sinc interpolation kernel. The DAF scaling function has beensuccessfully introduced as an efficient and powerful grid method forquantum dynamical propagations [40]. The Gaussian window in ourDAF-wavelets efficiently smoothes out the Gibbs oscillations, whichplague most conventional wavelet bases. The following equation shows theconnection between the B-spline function and the Gaussian window [34]:$\begin{matrix}{{\beta^{N}(x)} \cong {\sqrt{\frac{6}{\pi( {N + 1} )}}\quad\exp\quad( \frac{{- 6}\quad x^{2}}{N + 1} )}} & (236)\end{matrix}$for large N. As in FIG. 76, if we choose the window widthσ=η√{square root over ((N+1)/12)}  (237)the Gaussian Sinc wavelets generated by the lifting scheme similar tothe B-spline Sinc wavelets. Usually, the Gaussian Sinc DAF displays aslightly better smoothness and rapid decay than the B-spline Lagrangewavelets.

If we select more sophisticated window shapes, the Sinc wavelets can begeneralized further. We shall call these extensions Bell-windowed Sincwavelets. The available choice can be the different kinds of thepopularly-used DFT (discrete Fourier transform) windows, such asBartlett, Hanning, Hamming, Blackman, Chebechev, and Besel windows.

VISUAL GROUP NORMALIZATION

Filterbank Magnitude Normalization

On each decomposition level, 2-D wavelet coefficients are divided intofour sub-blocks, LL, HL, LH, and HH. As usual, L and H represent thelow-pass and high-pass subband filtering results, respectively. Forexample, HL means the signal passes the horizontal high-pass filterfirst and passes the vertical low-pass filter. Obviously, the detailedsub-blocks HL, LH and HH represent the multiscale difference operationin three different decomposition orientations, horizontal, vertical, anddiagonal, respectively. At each level of analysis, the bandwidth of theequivalent subband filters decreases by a factor of two.

Wavelet coefficients can be regarded as the output of the signal passingthrough equivalent decomposition filters (EDF). The responses of the EDFare the combination of several recurrent subband filters at differentstages [19, 28, 34, 37, 38, 40, 41]. The EDF amplitudes of varioussub-blocks differ greatly. Thus, the magnitude of the decompositioncoefficients in each of the sub-blocks cannot exactly reproduce theactual strength of the signal components. To adjust the magnitude of theresponse in each block, the decomposition coefficients are re-scaledwith respect to a common magnitude standard. standard. Thus EDFcoefficients, C_(j,m)(k), in block (j,m) should be multiplied with amagnitude scaling factor λ_(j,m), to obtain an adjusted magnituderepresentation. Here j represents the decomposition layer, and m denotesthe different orientation block (LL, LH, HL or HH).

We choose the normalizing factor to be the reciprocal of the maximummagnitude of the frequency response of the equivalent waveletdecomposition filter on node (j,m) $\begin{matrix}{{\lambda_{j,m} = \frac{1}{\sup\limits_{\omega \in \Omega}\{ {{{LC}_{j,m}(\omega)}} \}}}\quad,\quad{\Omega = \lbrack {0,{2\quad\pi}} \rbrack}} & (238)\end{matrix}$Thus, the magnitude normalized coefficients, NC_(j,m)(k), are defined asNC _(j,m)(k)=γ_(j,m) C _(j,m)(k)   (239)This idea was recently extended to Group Normalization (GN) of waveletpackets for signal processing [37, 38, 40, 41] and was shown to yieldthe optimal performance.

Filterband magnitude normalization (FMN) unifies the coefficientstrength in each of the subblocks. However, an image is a perceptualsignal source. Coefficients with equal magnitude (after FMN) indifferent frequency channels result in greatly different visual gain(sensitivity) for human eyes. Additional adjustments of the waveletcoefficients are required for visual image processing.

Perceptual Lossless Normalization

The reconstruction visibility of wavelet transform coefficients willdepend upon the display visual resolution [39] in pixel/degree. Given aviewing distance V in inches and a display resolution d in pixel/inch,the effective display visual resolution (DVR) R in pixel/degree ofvisual angle isR=dVtan(π/180)≈dV/57.3   (240)The visual resolution is the viewing distance in pixels (dV) divided by57.3.

As mentioned above, when the decomposition layer increases, thebandwidth of the equivalent subband filters decreases by a factor oftwo. Thus the frequency resolution doubles. Correspondingly, the space(time) resolution (display resolution) decreases by a factor of two. Asub-block of wavelet coefficients corresponds to a spatial frequencyband. For a display resolution of R pixel/degree, the spatial frequencyf of level j is

 ƒ=2^(−j) R   (241)

For a Y-channel gray-scale mammogram image, the just-noticeablequantization threshold of Y is generally different at each spatialfrequency. The contrast sensitivity declines when the spatial frequencyincreases (whereas, the size of the stimuli decreases). This fact isused to construct the “perceptual lossless” response magnitude fornormalization in subblock (j,m) according to the visual response.

For images based on the human vision system (HVS), using ajust-noticeable distortion profile, we can efficiently remove the visualredundancy from decomposition coefficients and normalize them withrespect to the standard of perceptual importance. A simple mathematicalmodel for perception efficiency, based on the amplitude nonlinearity indifferent frequency bands, has been presented in [39], which can be usedto construct the “perceptual lossless” response magnitude Y_(j,m) fornormalizing visual response in different luminance/chrominance spaces.We extend the definition to luminance/chrominance modes according to$\begin{matrix}{Y_{j,m} = {a\quad 10^{{k{({\log\quad\frac{2^{j}f_{0}d_{m}}{R}})}}^{2}}}} & (242)\end{matrix}$where a defines the minimum threshold, k is a constant, R is the DisplayVisual Resolution (DVR), ƒ₀ is the spatial frequency, and d_(m) is thedirectional response factor for each subblock.

The parameters d_(LL), and d_(HH) represent the thresholds fororientations LL and HL as frequency shifts relative to the threshold fororientations LH and HL. From the nature of dyadic wavelets, theorientation LL possesses a spectrum that is approximately a factor oftwo lower in spatial frequency than orientation LH or HL. This wouldsuggest a factor of d_(LL)=2. However, in a vision system responsemechanism, at orientation LL the signal energy is spread over all matrixelements, which implies less visual efficiency than when the energy isconcentrated over a narrow range, as is the case for the spectra oforientation HL or LH. Thus, the threshold should be increased, which canbe achieved by a slight reduction in ƒ₀. The final value for d_(LL) istherefore less than 2.

For orientation HH, similar effects exist. First, the Cartesiansplitting of the spectrum makes the spatial frequency of orientation HHabout √{square root over (2)} higher than that of the orientations HL,and LH. Thus d_(HH)can be taken as d_(HH)=2^(−½). As we mentionedbefore, the spectrum along orientation HH is spread over two orthogonalorientations (45° and 135°), which should result in a log thresholdincrease of about 2^(1/4) (a shift of 2^(−1/4)) or a total thresholdfactor of d_(HH)=2^(−3/4)=0.59. Finally, the well-known oblique effect[39], will cause a small amount of threshold elevation.

Visual Sensitivity Normalization

Visual sensitivity is defined as the inverse of the contrast required toproduce a threshold response [9],S=1/C   (243)where C is generally referred to simply as the threshold. The Michelsondefinition of contrast,C=(L _(max) −L _(mean))/L _(mean)   (244)is used, where L_(max) and L_(mean) refer to the maximum and meanluminances of the waveform in a luminance channel. Sensitivity can bethought of as a gain, although various nonlinearities of the visualsystem require caution in the use of this analogy. The variations insensitivity as a function of light level are primarily due to thelight-adaptive properties of the retina and are referred to as theamplitude nonlinearity of the HVS. The variations as a function ofspatial frequency are due to the optics of the eye combined with theneural circuitry; these combined effects are referred to as the contrastsensitivity function (CSF). Finally, the variations in sensitivity as afunction of signal content, referred to as masking, are due to thepost-receptoral neural circuitry.

Combining the perceptual lossless normalization, the visual sensitivitynormalization and the magnitude normalized factory λ_(jm), we obtain theperceptual lossless quantization matrix Q_(j,m)Q _(j,m)=2CY _(j,m)/λ_(j,m)   (245)This treatment provides a simple human-vision-based normalizationtechnique for the restoration of the most important perceptualinformation in a mammogram image. We call the combination of theabove-mentioned three normalizations the Visual Group Normalization(VGN) of wavelet transform coefficients.

IMAGE PROCESSING TECHNIQUES

Softer Logic Masking

The main objective of wavelet signal filtering is to preserve importantsignal components, and efficiently reduce noise components.

After visual group normalization processing, coefficients have beennormalized according to human visual sensitivity. An additionalfiltering algorithm is required for de-noising and removing perceptualredundancy. Hard logic masking is the commonly used nonlinear processingtechnique. It is similar to a bias estimated dead-zone limiter. Jain[14] has shown that a non-linear dead-zone limiter can improve the SNRfor weak signal detection. It can be expressed asη(y)=sgn(y)(|y|−δ)₊ ^(β), −1 ≦β≦1   (246)where δ is a threshold value, and y is the measurable value of thecoefficient. Donoho shows that the β=1 case of the above expression is anear optimal estimator for adaptive NMR data smoothing and de-noising[11]. Independently, we utilized the hard logic masking to efficientlyextract a target from formidable background noise in a previous work[26, 27, 28].

Various threshold cutoffs of multi-band expansion coefficients in hardlogic masking methods are very similar to the cutoff of an FFTexpansion. Thus, the Gibbs oscillations associated the FFT will alsooccur in the wavelet transform using a hard logic masking. Although hardlogic masking methods with appropriate threshold values do not seriouslychange the magnitude of a signal after reconstruction, they can causeconsiderable edge distortions in a signal due to the interference ofadditional high frequency components induced by the cutoff. The higherthe threshold value, the larger the Gibbs oscillations will be.

Edges are especially important for feature preservation and preciselocalization for images and biomedical signals. We here present a SofterLogic Masking (SLM) method. In our SLM approach, a smooth transitionband near each masking threshold is introduced so that any decompositioncoefficients, which are smaller than the threshold value, will bereduced gradually to zero rather than be set to zero. This treatmentefficiently suppresses Gibbs oscillations and preserves signal edges,and consequently improves the quality of the reconstructed signal. OurSLM method can be expressed as

 Ĉ _(j,m)(k)=sgn(C _(j,m)(k))(|C _(j,m)(k)−δ|)₊ ^(β)SOFT({overscore (NC_(j,m)(k))})   (247)

where the Ĉ_(j,m)(^(k)) are the decomposition coefficients to beretained in the reconstruction and the quantity {overscore (NCj,m(k))}is defined as $\begin{matrix}{\overset{\_}{{NC}_{j,m}(k)} = \frac{{{NC}_{j,m}(k)}}{\underset{{({j,m})} \in T_{opt}}{\max\quad}\{ {{{NC}_{j,m}(k)}} \}}} & (248)\end{matrix}$

The softer logic mapping, SOFT:[0,1]→[0,1], is a non-linearmonotonically increasing sigmoid functional. A comparison of the hardand softer logic masking functionals is depicted in FIG. 77.

The softer logic functional can also be taken as the alternated form$\begin{matrix}{{{\hat{C}}_{j,m}(k)} = {{C_{j,m}(k)}{{SOFT}( \frac{\overset{\_}{{NC}_{j,m}(k)} - \zeta}{1 - \zeta} )}}} & (249)\end{matrix}$where ζ is a normalized adaptive threshold. For an unknown noise level,an approximation to ζ is given asζ=γ_(upper){circumflex over (σ)}√{square root over (2logN/N)}  (250)where σ is a scaling factor and can be chosen as σ=1/1.349. The quantityγ_(upper) is an upper frame boundary of the wavelet packet transform,i.e., the upper boundary singular value of the wavelet transformdecomposition matrix. Using arguments similar to those given by Donoho[11], one can show that the above Softer Logic Masking reconstruction isa near optimal approximation in the min-max error sense.Device Adapted Enhancement

The basic idea is to use gradient operators to shape flat image data sothat desired portion of the image is projected onto a screen.

Using visual normalization and softer-logic thresholding, one canefficiently remove the visual redundancy and noisy components from thedecomposition coefficients.

For grayscale image contrast stretching, the objective is to improve theperception capability for image components which the human visual systemis initially insensitive but is important for diagnosis. In other words,mammogram enhancement increases the cancer detection probability andprecision. We first appropriately normalize the decompositioncoefficients according to the length scale of the display device so thatthey fall within the interval of the device frame. Assume that the imagecoefficients have already been properly scaled by visual groupnormalization so that the amplitude value NC_(j,m)(k) falls into thedynamic range of the display device:d _(min) ≦NC _(j,m)(k)≦d _(max)   (251)Without loss of generality, we consider the normalized gradientmagnitude,U _(j,m)(k)=NC _(j,m)(k)/(d _(max) −d _(min))   (252)

Mallat and Zhong realized that wavelet multiresolution analysis providesa natural characterization for multiscale image edges, and these can beeasily extracted by various differentiations [15]. Their idea wasextended by Laine et al [7] to develop directional edge parameters basedon a subspace energy measurement. An enhancement scheme based on complexDaubechies wavelets was proposed by Gagnon et al. [9]. These authorsmade use of the difference between the real and imaginary parts of thewavelet coefficients. One way or another, adjusted wavelet transformsshould be designed to achieve desired edge enhancement.

Our starting point is taken to be the magnitude normalized or visualgroup normalized wavelet subband coefficients NC_(j,m)(k) [10, 12]. Wedefine an enhancement functional E_(j,m)E _(j,m) =α _(j,m)+β_(j,m)Δ,   (253)where Δ is the Laplacian and −1≦α_(j,m), β_(j,m)≦1. The coefficientsα_(j,m), β_(j,m) can be easily chosen so that desired image features areemphasized. In particular we can emphasize an image edge of selectedgrain size. We note that a slight modification of α_(j,m) and β_(j,m)can result in orientation selected image enhancement. A detaileddiscussion of this matter will be presented elsewhere. An overallre-normalization is conducted after image reconstruction to preserve theenergy of the original image. We call this procedure enhancementnormalization.

Contrast stretching is an old but quite efficient method for featureselective image display. Nonlinear stretching has been used by manyauthors [3, 7, and 16]. Lu and coworkers [16] have recently designed ahyperbolic function

 g _(j)(k)=[tanh(ak−b)+tanh(b)]/[tanh(a−b)+tanh(b)]  (254)

for wavelet multiscale gradient transformation. Their method works wellfor lunar images. The basic idea is to use gradient operators to shape aflat image so that desired portion of the image is projected into ascreen window.

ENHANCEMENT RESULT

To test our new approaches, digital breast mammogram images areemployed. Mammograms are complex in appearance and signs of earlydisease are often small and/or subtle. Digital mammogram imageenhancement is particularly important for aiding radiologists in thedevelopment of an automatically detecting expert system. The originalimage is coded at 768×800 size and a 200-micron pixel edge as shown inFIG. 78(a). As shown in FIG. 78(b), there is a significant improvementin both edge representation and image contrast resulting fromDAF-wavelets combined with our Visual Group Normalization (VGN) andnon-linear enhancement techniques. In particular, the domain andinternal structure of higher-density cancer tissues are clearlydisplayed. The results are characterized by high-quality imageenhancement and good signal averaging over homogeneous regions withminimal resolution degradation of image details.

CONCLUSION

The newly developed DAF-wavelet image processing and enhancementtechniques can improve present and future imaging performance forearlier detection of cancer and malignant tumors. It improves thespatio-temporal resolution of biomedical images and enhances thevisualization of the perceptually less-sensitive components that arevery important for diagnosis, as well as reduces the distortion andblur.

Different image processing techniques (distortion suppression,enhancement, and edge sharpening) can be integrated in a single stepprocess or can be alternatively applied by different coefficientselection. Our biomedical image processing and computer vision researchshould prove important in meeting hospital needs in image enhancement,image analysis and expert diagnosis.

The method presented can be applied to various types of medical images.These include various X-ray images, Mammography, Magnetic ResonanceImaging (MRI), Supersonic imaging, etc. Enhanced imaging of internalorgans and/or other parts of the human body, for the detection of cancerand other diseases, is of great importance. It has the potential forearlier, cost-effective diagnosis and management of disease, can provideimproved visualization of normal versus diseased tissue, and enhance thestudy of drug diffusion and cellular uptake in the brain.

The method dramatically improves the image quality (in terms of signalto noise improvements and/or contrast differentiation) by de-noising andenhancement, as demonstratedly the digital mammogram example.

REFERENCES

-   [1] Ansari, C. Guillemot, and J. F. Kaiser, “Wavelet construction    using Lagrange halfband filters,” IEEE Trans. CAS, vol. 38, no. 9,    pp. 1116-1118, 1991.-   [2] R. Baraniuk, D. Jones, “Signal-dependent time-frequency analysis    using a radially Gaussian kernel,” Signal Processing, Vol. 32, pp.    263-284, 1993.-   [3] C. M. Brislawn, “Preservation of subband symmetry in multirate    signal coding,” IEEE Trans. SP, vol. 43, no. 12, pp. 3046-3050,    1995.-   [4] C. K. Chui, An Introduction to Wavelets, Academic Press, New    York, 1992.-   [5] C. K. Chui, Wavelets: A Tutorial in Wavelet Theory and    Applications, Academic Press, New York, 1992.-   [6] I. Daubechies, “Orthonormal bases of compactly supported    wavelets”, Comm. Pure and Appl. Math., vol. 41, no. 11, pp. 909˜996,    1988.-   [7] I. Daubechies, “The wavelet transform, time-frequency    localization and signal analysis,” IEEE Trans. Inform. Theory, Vol.    36, No. 5, pp. 961-1003, September 1990.-   [8] G. Deslauriers, S. Dubuc, “Symmetric iterative interpolation    processes,” Constructive Approximations, vol. 5, pp. 49-68, 1989.-   [9] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.    Information Theory, vol. 41, no. 3, pp. 613˜627, 1995.-   [10] D. L. Donoho, “Interpolating wavelet transform,” Preprint,    Stanford Univ., 1992.-   [11] S. Dubuc, “Interpolation through an iterative scheme”, J. Math.    Anal. and Appl., vol. 114, pp. 185˜204, 1986.-   [12] A. Harten, “Multiresolution representation of data: a general    framework,” SIAM J. Numer. Anal., vol. 33, no. 3, pp. 1205-1256,    1996.-   [13] C. Herley, M. Vetterli, “Orthogonal time-varying filter banks    and wavelet packets,” IEEE Trans. SP, Vol. 42, No. 10, pp.    2650-2663, October 1994.-   [14] C. Herley, Z. Xiong, K. Ramchandran and M. T. Orchard, “Joint    Space-frequency Segmentation Using Balanced Wavelet Packets Trees    for Least-cost Image Representation,” IEEE Trans. Image Processing,    vol. 6, pp. 1213-1230, September 1997.-   [15] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri,    “Analytic banded approximation for the discretized free    propagator,” J. Physical Chemistry, vol. 95, no. 21, pp. 8299-8305,    1991.-   [16] L. C. Jain, N. M. Blachman, and P. M. Chapell, “Interference    suppression by biased nonlinearities,” IEEE Trans. IT, vol. 41, no.    2, pp. 496-507, 1995.-   [17] N. Jayant, J. Johnston, and R. Safranek, “Signal compression    based on models of human perception”, Proc. IEEE, vol. 81, no. 10,    pp. 1385˜1422, 1993.-   [18] J. Kovacevic, and M. Vetterli, “Perfect reconstruction filter    banks with rational sampling factors,” IEEE Trans. SP, Vol. 41, No.    6, pp. 2047-2066, June 1993.-   [19] J. Kovacevic, W. Swelden, “Wavelet families of increasing order    in arbitrary dimensions,” Submitted to IEEE Trans. Image Processing,    1997.-   [20] A. F. Laine, S. Schuler, J. Fan and W. Huda, “Mammographic    feature enhancement by multiscale analysis,” IEEE Trans. MI, vol.    13, pp. 725-740, 1994.-   [21] S. Mallat, “A theory for multiresolution signal decomposition:    the wavelet representation,” IEEE Trans. PAMI, Vol. 11, No. 7, pp.    674-693, July 1989.-   [22] Y. Meyer, Wavelets Algorithms and Applications, SIAM Publ.,    Philadelphia 1993.-   [23] K. Ramchandran, M. Vetterli, “Best wavelet packet bases in a    rate-distortion sense,” IEEE Trans. Image Processing, Vol. 2, No. 2,    pp. 160-175, April 1993.-   [24] K. Ramchandran, Z. Xiong, K. Asai and M. Vetterli, “Adaptive    Transforms for Image Coding Using Spatially-varying Wavelet    Packets,” IEEE Trans. Image Processing, vol. 5, pp. 1197-1204, July    1996.-   [25] O. Rioul, M. Vetterli, “Wavelet and signal processing,” IEEE    Signal Processing Mag., pp. 14-38, October 1991.-   [26] N. Saito, G. Beylkin, “Multiscale representations using the    auto-correlation functions of compactly supported wavelets,” IEEE    Trans. Signal Processing, Vol. 41, no. 12, pp. 3584-3590, 1993.-   [27] M. J. Shensa, “The discrete wavelet transform: wedding the a    trous and Mallat algorithms”, IEEE Trans. SP, vol. 40, no. 10, pp.    2464˜2482, 1992.-   [28] Z. Shi, Z. Bao, “Group-normalized processing of complex wavelet    packets,” Science in China (Serial E), Vol. 26, No. 12, 1996.-   [29] Z. Shi, Z. Bao, “Group-normalized wavelet packet signal    processing”, Wavelet Application IV, SPIE, vol. 3078, pp. 226˜239,    1997.-   [30] Z. Shi, Z. Bao, “Fast image coding of interval interpolating    wavelets,” Wavelet Application IV, SPIE, vol. 3078, pp. 240-253,    1997.-   [31] Z. Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Perceptual    image processing using Gauss-Lagrange distributed approximating    functional wavelets,” submitted to IEEE SP Letter, 1998.-   [32] W. Swelden, “The lifting scheme: a custom-design construction    of biorthogonal wavelets,” Appl. And Comput. Harmonic Anal., vol. 3,    no. 2, pp. 186˜200, 1996.-   [33] T. D. Tran, R. Safranek, “A locally adaptive perceptual masking    threshold model for image coding,” Proc. ICASSP, 1996.-   [34] M. Unser, A. Adroubi, and M. Eden, “The L₂ polynomial spline    pyramid,” IEEE Trans. PAMI, vol. 15, no. 4, pp. 364-379, 1993.-   [35] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part I: system-theoretic fundamentals,” IEEE    Trans. SP, Vol. 43, No. 5, pp. 1090-1102, May 1995.-   [36] P. Vaidyanathan, T. Chen, “Role of anti-causal inverse in    multirate filter-banks—Part II: the FIR case, factorizations, and    biorthogonal lapped transforms,” IEEE Trans. SP, Vol. 43, No. 5, pp.    1103-1115, May 1995.-   [37] M. Vetterli, C. Herley, “Wavelet and filter banks: theory and    design,” IEEE Trans. SP, Vol. 40, No. 9, pp. 2207-2232, September    1992.-   [38] J. D. Villasenor, B. Belzer, and J. Liao, “Wavelet filter    evaluation for image processing,” IEEE Trans. IP, vol. 4, no. 8, pp    1053-1060, 1995.-   [39] A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor,    “Visibility of wavelet quantization noise,” IEEE Trans. Image    Processing, vol. 6, pp. 1164-1175, 1997.-   [40] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,    “Lagrange distributed approximating Functionals,” Physical Review    Letters, Vol. 79, No. 5, pp. 775˜779, 1997.-   [41] G. W. Wei, D. J. Kouri, and D. K. Hoffman, “Wavelets and    distributed approximating functionals,” submitted to Phys. Rev.    Lett.-   [42] Z. Xiong, K. Ramchandran and M. T. Orchard, “Space-frequency    Quantization for Wavelet Image Coding,” IEEE Trans. Image    Processing, vol. 6, pp. 677-693, May 1997.-   [43] S. H. Zhang, Z. Bao, etc. “Target extraction from strong    clutter background,” Tech. Rep., National Key Lab. of Radar Signal    Processing, Xidian University, 1994

All references (articles and patents) referenced or cited in thisdisclosure are incorporated herein by reference. Although the inventionhas been disclosed with reference to its preferred embodiments, fromreading this description those of skill in the art may appreciatechanges and modification that may be made which do not depart from thescope and spirit of the invention as described above and claimedhereafter.

DUAL PROPAGATION INVERSION OF FOURIER AND LAPLACE SIGNALS Introduction

In many physical and chemical phenomena, two functions are related by anintegral equation of the formC(x)=∫dx′K(x,x′)ƒ(x′)   (255)where K(x,x′) is some sort of “Transform Kernel”, and x, x′ may beconjugate variables (depending on the transform under consideration).For experimental or computational studies, the function C(x) istypically known only on a finite, discrete set of points. Extractingƒ(x′) from measured or calculated values of C(x) is referred to as“inversion”. Famous examples of this inversion problem are the Fourierand Laplace transforms. Although there are well-known, exactmathematical procedures for these inversions, one generally facesserious difficulties in numerically determining the function f(x′) fromC(x)-values known at a finite number of discrete points. In the Laplacetransform, for example, the inversion may be unstable due toamplification of noise or other errors in the C(x_(i)) data [1]. InverseFourier transforms can also suffer from slow convergence of theintegration, requiring a large range of samples of the C(x_(i)), so thatone is able to obtain only low resolution spectra when a sufficientlylong duration time signal is not available [2,3].

A convenient procedure for numerically inverting such an integralequation is disclosed, with the focus here being primarily on real-timespectra. We shall demonstrate the method using a simple Fouriertransformation as an example, but the procedure may be useful for otherinversion problems. Our method, termed the “dual propagationinversion”(DPI) method, makes use of the distributed approximatingfunctional (DAF)free propagator [4] to carry out the inversion, butother numerical free propagator techniques (such as the Fast Fouriermethod [5,6]) may also be used. The Fourier transform is naturallyinvoked whenever one tries to obtain information in the frequency-domainfrom a time-domain signal. In general, for real spectra, the time-domainand frequency-domain spectra are related by $\begin{matrix}{{C(t)} = {\int_{- \infty}^{\infty}\quad{{\mathbb{d}\omega}\quad{\mathbb{e}}^{{- i}\quad\omega\quad t}{f(\omega)}}}} & (256)\end{matrix}$and $\begin{matrix}{{f(\omega)} = {\frac{1}{2\quad\pi}{\int_{- \infty}^{\infty}\quad{{\mathbb{d}t}\quad{\mathbb{e}}^{i\quad\omega\quad t}{C(t)}}}}} & (257)\end{matrix}$Typical examples of the sorts of problems of interest are thecomputation of absorption spectra or Raman spectra [7] from theautocorrelation function and the extraction of vibrational frequenciesfrom a molecular dynamics simulated time signal [8]. The difficulty inthe direct inversion by Equation (257) is that in order to resolveclose-lying frequencies, one needs to know the autocorretation function(or time signal in general) on a very dense time grid over a long periodof time [2,3]. When the time signal is available only on a set ofdiscrete times and for a short time period, this approach becomesinefficient and inaccurate. Several elegant methods, such as thefilter-diagonalization technique, developed by Neuhauser [9] andmodified by Mandelshtam and Taylor [10], have been proposed for suchproblems.

In the present approach, we begin by inserting a factor, exp[iα(ω−ω₀)²],into Equation (256), thereby defining an auxiliary function$\begin{matrix}{{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )} = {\int_{- \infty}^{\infty}\quad{{\mathbb{d}\omega}\quad{\mathbb{e}}^{{- i}\quad{\alpha{({\omega - \omega_{0}})}}^{2}}{\mathbb{e}}^{{- i}\quad\omega\quad t}{{f(\omega)}.}}}} & (258)\end{matrix}$(Note that Equation (258) can be viewed as a type of Fresnel transform[11] an integral kernel K(ω,ω₀)=exp[iα(ω−ω₀)²]). From that viewpoint,the kernel acts on e^(−iωt)ƒ(ω) rather than on ƒ(ω).) Due to the highlyoscillatory nature of the factor exp[iα(ω−ω₀)²] for large α values, onlyfrequencies sufficiently close to ω=ω₀ contribute significantly toEquation (258). In fact, it can be shown that in the limit α→∞, {hacekover (C)}(t;′α,ω₀) becomes proportional to ƒ(ω). The factor,exp[iα(ω−ω₀)²], can be viewed as a window function that acts as a“filter” so that the auxiliary function, {hacek over (C)}(t;α,ω₀),contains information principally at frequencies near ω=ω₀. We note thatthe factor exp[iα(ω−ω₀)²] also can be expressed as a matrix element ofthe “free propagator”, exp[(−i/4α)(d²/dω²)], according to$\begin{matrix}{{\mathbb{e}}^{i\quad\alpha\quad{({\omega - \omega_{0}})}^{2}} = {{( {\pi\quad{i/\alpha}} )^{1/2}\langle {\omega{{\mathbb{e}}^{{\frac{- i}{4\quad\alpha}\quad\frac{\mathbb{d}^{2}}{\mathbb{d}\quad\omega^{2}}}\quad}}\omega_{0}} \rangle} = {( {\pi\quad{i/\alpha}} )^{1/2}{\mathbb{e}}^{{\frac{- i}{4\quad\alpha}\quad\frac{\mathbb{d}^{2}}{\mathbb{d}\quad\omega_{0}^{2}}}\quad}{\delta( {\omega - \omega_{0}} )}}}} & (259)\end{matrix}$and the integration over ω in Equation (258) is equivalent to evaluatingthe action of exp[(−i/4α)(d²/dω²)] in the continuous “ω”-representation.That is, the introduction of the phase exp[iα(ω−ω₀)²] is equivalent tofreely propagating (in the sense of quantum propagation) the functione^(−iωt)ƒ(ω) for a “duration”, 1/(4α), in a generalized time. Thisenables us to write Equation (258) as $\begin{matrix}{{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )} = {( {\pi\quad{i/\alpha}} )^{1/2}{\exp\lbrack {{- \frac{i}{4\quad\alpha}}\quad\frac{\mathbb{d}^{2}}{\mathbb{d}\quad\omega_{0}^{2}}} \rbrack}( {{\mathbb{e}}^{{- i}\quad\omega_{0}t}{f( \omega_{0} )}} )}} & (260)\end{matrix}$The function ƒ(ω₀) then is obtained simply by inverting Equation (260),$\begin{matrix}{{f( \omega_{0} )} = {( {{\alpha/\pi}\quad i} )^{1/2}{\mathbb{e}}^{i\quad\omega_{0}t}{\mathbb{e}}^{{\frac{i}{4\quad\alpha}\quad\frac{\mathbb{d}^{2}}{\mathbb{d}\quad\omega_{0}^{2}}}\quad}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}}} & (261)\end{matrix}$This equation indicates that the value of the function ƒ(ω₀) can becomputed by freely propagating the auxiliary function {hacek over(C)}(t;α) for a “duration” 1/(4α). However, for this equation to be ofany use, we need an independent method for calculating the auxiliaryfunction from the experimentally (or theoretically) determined C(t). Thefunction {hacek over (C)}(t;α,ω₀) can be calculated directly from theexperimentally determined signal, C(t), using the following expression:$\begin{matrix}{{\overset{\sim}{C}( {{t_{0};\alpha},\omega_{0}} )} = {{\mathbb{e}}^{{- i}\quad\omega_{0}t_{0}}{{\mathbb{e}}^{{- i}\quad\alpha\frac{\mathbb{d}^{2}}{\mathbb{d}\quad t_{0}^{2}}}\lbrack {{\mathbb{e}}^{i\quad\omega_{0}t_{0}}{C( t_{0} )}} \rbrack}}} & (262)\end{matrix}$This equation is obtained by deriving (and integrating) the auxiliarypartial differential equation, $\begin{matrix}{{\frac{\partial}{\partial\alpha}\lbrack {{\mathbb{e}}^{i\quad\omega_{0}t}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}} \rbrack} = {{- i}\quad{\frac{\partial^{2}}{\partial t^{2}}\lbrack {{\mathbb{e}}^{i\quad\omega_{0}t}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}} \rbrack}}} & (263)\end{matrix}$Thus, our procedure consists of propagating the experimental timesignal, C(t) (multiplied by exp(iω₀t)), from t t₀ to over the“duration”α to obtain {hacek over (C)}(t₀;α,ω₀), and then propagating theresulting {hacek over (C)}(t₀;α,ω₀) from %ω₀ to ω over the “duration”1/(4α). Clearly, the product of the two “propagation durations” is aconstant (α×¼α=¼), indicating that there exists a reciprocalrelationship characteristic of a “time-energy”-like uncertaintyprinciple. In the numerical example we give below, we shall chooseα=¼α=½; however, this is not necessary and the question of the optimum“duration” for each propagation remains to be explored. We also find itnumerically efficient to employ the DAF-free propagator [4],$\begin{matrix}{\langle {\omega{{\mathbb{e}}^{{i\quad\alpha\quad\frac{\mathbb{d}^{2}}{\mathbb{d}\quad\omega^{2}}}\quad}}\omega_{0}} \rangle = {\sum\limits_{n = 0}^{M/2}{{b_{n}( \frac{\sigma(0)}{\sigma(\alpha)} )}^{{2n} + 1}{\exp\lbrack {- \frac{( {\omega - \omega_{0}} )^{2}}{2\quad{\sigma^{2}(\alpha)}}} \rbrack}{H_{2n}\lbrack {- \frac{( {\omega - \omega_{0}} )}{\sqrt{2}{\sigma(\alpha)}}} \rbrack}}}} & (264)\end{matrix}$whereb _(n)=(−1)^(n)/[(2π)^(1/2)σ(0)n!2^(2n])  (264)andσ²(α)=σ²(0)+2iα  (266)where, σ(0) and M are DAF-parameters, and the H_(2n)'s are the Hermitepolynomials [4,12]. This makes it possible to carry out the twopropagations with a single matrix-vector multiplication for eachpropagation. It should be noted that this DAF-free propagator exactlypropagates the DAF representation of a function.

We believe that the dual-propagation inversion scheme we have outlinedhas several potential advantages. First, the DAF-fitting method on whichthe scheme is based allows us to effectively filter noise whilepreserving signal. The method provides a “signature” as a function ofthe DAF parameters which gives a measure of signal to noise and allowsthe parameters to be varied to achieve optimum filtering. Second, asalready noted, the propagation of the DAF representation of a functionusing the Hermite DAF propagator is exact. Third, each propagation hasassociated with it a characteristic width controlled by the Gaussian(see Equations (264) and (266)) and determined by the optimum fit of theinput data. These widths control on what domain of t the function C(t)must be known in order to determine a particular value of ƒ(ω₀).Finally, DAFs can be used to “pad” (i.e., extend) the initial C(t)dataset in various ways to take advantage of any a priori knowledge onemight have.

A detailed comparison of our method with other inversion methods isbeyond the scope of this brief report and will be the topic of asubsequent communication. However, we will illustrate the method on achallenging problem, namely extracting a half-sine wave from itsdiscretized and noisy transform. The underlying function, ƒ(ω), in ourexample is then ${f(\omega)} = \{ \begin{matrix}{{\sin\quad(\omega)},} & {\lbrack {0 \leq \omega \leq \pi} \rbrack\quad(267)} \\{0,} & {otherwise}\end{matrix} $Since ƒ(ω) has compact support, the time-signal, C(t)(analyticallyobtained by Equation (256)) of course, cannot have compact support. Infact, it has a very slow decay with increasing |t|. First, θ(ω) wascomputed by our dual propagation using Equations (261) and (262), andthe result compared with the truncated sine function, Equation (267). InFIG. 79, we present the auxiliary function, {hacek over (C)}(t;α,ω₀), att=0 and α={fraction (1/2 )}, as a function of ω₀. We have calculatedthis function both by direct numerical integration of Equation (258) andby DAF propagation using Equation (262) [13]. The results of the DAFpropagation and the direct numerical integration are essentiallyidentical up to the fourth decimal point. The propagation to generatethe auxiliary function has been carried out by a highlyefficient“one-step DAF-propagation” of duration α=½. FIG. 80 comparesthe spectral function, ƒ(ω), obtained by the present dual propagation[13] with the original truncated sine-function. It Asian that the twoare visually indistinguishable. In this calculation, we assumed discrete“experimental” values of C(t) are known from −45≦t≦45 on a discrete gridwith uniform spacing Δt₀={fraction (1/18)}, and the propagation from ω₀to ω employs the computed values of {hacek over (C)}(t₀;α,ω₀) from−30≦ω₀≦35, with a uniform grid spacing of Δω₀={fraction (1/18)}. Weemphasize that one only needs the time signal C(t) for a finite timeinterval, [t_(min),t_(max)], in order to calculate {hacek over(C)}(t₀;α,ω₀) using Equation (262). Although the accuracy increases forlonger-time signals, C(t), the length of the time interval is controlledthe width of the DAF free propagator matrix (determined by σ/Δt₀). Thus,short-duration time signals can yield an accurate spectrum ƒ(ω) for thefrequencies filtered out in Equations (258) and (262) in our method. Itshould also be noted that the present technique does not require aknowledge of the system Hamiltonian, or any diagonalization processes,since it makes direct use of the time signal (or autocorrelationfunction). Thus, the measured or computed time signals directly yieldinformation in the frequency domain (e.g., optical spectra or normalmode frequencies).

One of the features of DAFs that has been discussed previously is theirability to filter noise from a given function. The free-propagating DAFdescribed here also has this filtering property, which is extremelyuseful in the present context. When an inversion is carried out forexperimental data, the presence of noise in the time signal C(t) willcreate inaccuracies in the calculated spectrum, ƒ(ω). (This problem isespecially serious in the inverse Laplace transformation, sincestatistical errors present in the signal can be easily amplified.) Inorder to illustrate the filtering feature of our DPI procedure, we addedrandom noise(up to ±20\%) to the time signal C(t), and carried out thedual propagation inversion as before. We find that the resultingauxiliary function, {hacek over (C)}(t₀;α,ω₀) is very similar to thatobtained earlier using the noise-free time signal, confirming that highfrequency, random noise is automatically removed by the DPI method. FIG.81 illustrates this denoising feature of the DPI, using Hermite-DAF freepropagators, by comparing the spectrum obtained from the “corrupted”time signal (with±20% noise) to that obtained from the noise-free timesignal. Both time domain signals extend over the range −45≦t≦45, withthe same sampling frequency as before. It is seen that the DPI almostcompletely removes the effects of the noise in the time signal. We alsoemployed a stationary DAF filtering method [14] to remove the noise fromthe time signal before carrying out the DPI procedure, and obtained afrequency domain spectrum ƒ(ω) that is visually indistinguishable fromthat depicted in FIG. 81.

Finally, we explore the consequences of a more severe truncation of thetime-domain signal, and therefore carried out the DPI procedure usingC(t) over the artificially shortened interval −5≦t≦5. In thesecalculations, we have treated the noiseless C(t). Our results are shownin FIG. 82, where we compare the inversion result to the exact, originalsignal. We see that there is relatively little error induced (comparableto that due to introducing noise; see FIG. 81). A common procedure todecrease the aliasing effects caused by the truncation of the timedomain signal is to artificially damp C(t) to zero beyond certain times.This is typically done by interpolation or by smoothly joining anexponential decaying analytical “tail”. In the case of DAFs, we can adda gap on either side of the truncated C(t), and use DAF-fitting to “fillin” the missing gap values, thereby joining the actual data to ananalytical tail function. In the present study, we introduced the gaps5≦|t|≦7.5, and joined the known data between −5≦t≦5 to the analyticaltailC(t)=−0.040648exp(−0.107871|t|)   (268)The results of the DPI are compared to the original signal in FIG. 83,and we see that while some aliasing still exists, it is reduced over theentire range of ω. Similar results are obtained when noisy signals aretruncated, and then DAF-joined to an analytical decaying tail function.We conclude by noting that we expect even better results will beobtained by using DAFs to periodically (or in other ways) extend thetruncated time domain signal. This will be reported elsewhere.

We conclude by briefly considering the Laplace transform problem, namelynumerically inverting $\begin{matrix}{{C(t)} = {\int_{0}^{\infty}\quad{{\mathbb{d}\omega}\quad{\mathbb{e}}^{{- \omega}\quad t}{f(\omega)}}}} & (269)\end{matrix}$which our method (in principle) also solves. Here C(t) is analytic inthe half-plane (t>0). In the standard way, it is convenient to introducea new function ${\overset{\_}{f}(\omega)} = \{ \begin{matrix}{{{f(\omega)}\quad,}\quad} & {{{for}\quad\omega} \geq 0} \\{{0,}\quad} & {{{for}\quad\omega} < 0}\end{matrix} $to write $\begin{matrix}{{C(t)} = {\int_{- \infty}^{\infty}\quad{{\mathbb{d}\omega}\quad{\mathbb{e}}^{{- \omega}\quad t}{\overset{\_}{f}(\omega)}}}} & (270)\end{matrix}$Exactly paralleling our previous discussion we then obtain$\begin{matrix}{{\overset{\_}{f}( \omega_{0} )} = {( {{\alpha/\pi}\quad i} )^{1/2}{\mathbb{e}}^{\omega_{0}t}{\mathbb{e}}^{\frac{i}{4\quad\alpha}\quad\frac{\mathbb{d}^{2}}{\mathbb{d}\quad\omega_{0}^{2}}}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}}} & (271)\end{matrix}$where now $\begin{matrix}{{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )} = {\int_{- \infty}^{\infty}\quad{{\mathbb{d}\omega}\quad{\mathbb{e}}^{i\quad{\alpha{({\omega - \omega_{0}})}}^{2}}{\mathbb{e}}^{{- \omega}\quad t}{\overset{\_}{f}(\omega)}}}} & (272)\end{matrix}$The quantity {hacek over (C)}(t;α,ω₀) obeys the equation $\begin{matrix}{{\frac{\partial}{\partial\alpha}\lbrack {{\mathbb{e}}^{\omega_{0}t}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}} \rbrack} = {{- i}\quad{\frac{\partial^{2}}{\partial t^{2}}\lbrack {{\mathbb{e}}^{\omega_{0}t}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}} \rbrack}}} & (273)\end{matrix}$subject to the initial-value boundary condition {hacek over(C)}(t;α=0,ω₀)=C(t). Equation (273) is identical to Equation (263)except for the sign of the right-hand side. The inversion then proceedsby solving Equation (273), subject to the boundary condition at α=0 andpropagating the result via Equation (271) to obtain the answer. Althoughthis solves the problem in principle, the first step(that of solvingEquation (273) to acceptable accuracy when C(t) is imprecisely known) isa challenge. In the Fourier transform case, we solved the correspondingpartial differential equation by means of a second propagation, andindeed we can still write the formal solution to Equation (273) as$\begin{matrix}{{{\mathbb{e}}^{\omega_{0}t}{\overset{\sim}{C}( {{t;\alpha},\omega_{0}} )}} = {{{{\mathbb{e}}^{i\quad\alpha\frac{\partial}{\partial t}}\lbrack {{\mathbb{e}}^{\omega_{0}t}{C(t)}} \rbrack}\quad{for}\quad t} > 0}} & (274)\end{matrix}$since C(t) is analytic in the half-plane (t>0) and hence its derivativesto all orders uniquely exist in the half-plane. However, using the DAFpropagation scheme (at least in a straight-forward manner) presentsdifficulties near the origin because of the singularity in C(t) at t=0.Various stratagems can be devised for attacking these problems, and theyare currently under investigation.

REFERENCES

-   [1] E. Gallicchio and B. J. Berne, {\it J. Chem. Phys.101, 9909    (1994); ibid. 105, 7064 (1996).-   [2] M. R. Wall and D. Neuhauser, J. Chem. Phys. 102, 8011 (1995).-   [3] V. A. Mandelshtam and H. S. Taylor, J. Chem. Phys. 106, 5085    (1997).-   [4] D. K. Hoffman, M. Arnold, and D. J. Kouri, J. Phys. Chem. 97,    1110 (1993); D. J. Kouri and D. K. Hoffman, in {\it Time-Dependent    Quantum Molecular Dynamics}, eds. J. Broeckhove and L. Lathouwers    (Plenum Press, New York, N.Y., 1992); see also N. Nayar, D. K.    Hoffman, X. Ma, and D. J. Kouri, {\it J. Phys. Chem. {\bf 96}, 9637    (1992).-   [5] H. J. Nussbaumer, {\it Fast Fourier Transform and Convolution    Algorithms}, 2nd. ed. (Springer Verlag, Berlin, 1982).-   [6] E. O. Brigham, {\it The Fast Fourier Transform (Prentice-Hall,    Englewood Cliffs, N.J., 1974).-   [7] E. J. Heller, {\it Accnts. Chem. Res.} {\bf14}, 368 (1981); D.    Imre, J. L. Kinsey, A. Sinha, and J. Krenos, {\it J. Phys. Chem.}    {\bf 88}, 3956 (1984); E. J. Heller, R. L. Sundberg, and D. Tannor,    {\it J. Phys. Chem.} {\bf86}, 1822 (1982); S.-Y. Lee and E. J.    Heller, {\it J. Chem. Phys.} {\bf71}, 4777 (1979).-   [8] M. D. Feit, J. A. Fleck, and A. Steiger, {\it J. Comp. Phys.}    {\bf 47}, 412 (1982); E. J. Heller, E. B. Stechel, and M. J. Davis,    {\it J. Chem. Phys.} {\bf73}, 4720 (1980).-   [9] D. Neuhauser, {\it J. Chem. Phys.} {\bf93}, 2611 (1990); {\it    ibid.} {\bf 100}, 5076 (1994).-   [10] V. A. Mandelshtam and H. S. Taylor, {\it Phys. Rev. Lett.}    {\bf78}, 3274(1997); {\it J. Chem. Phys.} {\bf 107}, 6756 (1997).-   [11] F. Gori, {\it Optics Commun.} {\bf 39}, 293 (1981); A.    Papoulis, {\it Systems and Transforms with Applications to Optics}    (McGraw-Hill, New York, N.Y., 1968); L. Mertz, {\it Transformations    in Optics} (Wiley, New York, N.Y., 1965).-   [12] D. K. Hoffman, N. Nayar, O. A. Sharafeddin, and D. J. Kouri,    {\it J. Phys. Chem.} {\bf95}, 8299 (1991); D. J. Kouri, W. Zhu, X.    Ma, B. M. Pettitt, and D. K. Hoffman, {\it ibid.} {\bf 96}, 9631    (1992); D. K. Hoffman and D. J. Kouri, in {\it Proc. 3rd. Int. Conf.    Math. and Num. Aspects of Wave Prop.}, ed. G. Cohen (SIAM,    Philadelphia, Pa., 1995) pp. 56-83; M. Arnold, D. K. Hoffmnan,    and D. J. Kouri, unpublished.-   [13] The parameters employed are −45≦t_(—)0≦45, M=40, σσΛDelta t=2.5    and \Delta t={fraction (1/18)}.-   [14] D. K. Hoffman, G. H. Gunaratne, D. S. Zhang, and D. J. Kouri,    “Fourier Filtering of Images”, {\it CHAOS}, submitted; D. S.    Zhang, D. J. Kouri, D. K. Hoffman, and G. H. Gunaratne, “Distributed    Approximating Functional Treatment of Noisy Signals”, to be    published; D. S. Zhang, Z. Shi, G. W. Wei, D. J. Kouri, G. H.    Gunaratne, and D. K. Hoffman, “Distributed Approximating Functional    Approach to Image Restoration”, to be published.

DISTRIBUTED APPROXIMATING FUNCTIONAL WAVELET NETS Introduction

For real-world signal processing, pattern recognition and systemidentification, information extraction from a noisy background is thefundamental objective. To obtain an ideal output vector Y(X, W) from theobservation input vector X, the system (neural network) should possessthe following two kinds of smoothness [6] (where W is the response orthe entry of the system, normally for neural networks, called the weightvector).

(a) Functional space smoothness

(b) State space smoothness

The degree of smoothness in functional space governs quality of thefiltering of the observed (noisy) signal. The smoother the outputsignal, the more noise is suppressed. State space smoothness impliesthat a weak fluctuation of the weight vector W={w(i), i=0, L−1}, has asmall effect on the output signal, which makes the system less sensitiveto the input distortion.

For a robust estimation system, the output not only approaches theobserved signal value, but also smoothes the signal to suppress thedistortion due to noise. Simultaneously, the state space should besmooth to ensure stability. Based on these facts, one finds that theleast mean square (LMS) errorE _(A) =∫[Y(X)−Ŷ(W, X)]² dX.   (275)the regularization constraints of order r $\begin{matrix}{E_{R} = {\int{\lbrack \frac{\partial^{r}{\hat{Y}( {W,X} )}}{\partial X^{r}} \rbrack^{2}{\mathbb{d}X}}}} & (276)\end{matrix}$and the condition of the system $\begin{matrix}{E_{W} = \frac{W}{\sum\limits_{i}{{g( x_{i} )}}^{2}}} & (277)\end{matrix}$are the three dominant factors that must be accounted for in designing arobust estimation system.

Distributed Approximating Functionals (DAFs), which can be constructedas a window modulated interpolating shell, were introduced previously asa powerful grid method for numerically solving partial differentialequations with extremely high accuracy and computational efficiency[3,9,10]. In this paper, we use the DAF approximation scheme forimplementing a neural network.

Compared with other popular networks, DAF nets possess advantages inseveral areas:

-   -   (1) a DAF wavelet is infinitely smooth in both time and        frequency domains.    -   (2) For essentially arbitrary order of the Hermite polynomial,        the DAF shells possess an approximately constant shape, while        commonly used wavelet functions always become more oscillatory        as the regularization order is increased.    -   (3) The translation invariance of the DAF approximation ensures        feature preservation in state space. The signal processing        analysis can be implemented in a space spanned by the DAFs        directly.    -   (4) Complicated mathematical operations, such as differentiation        or integration, can be carried out conveniently using the DAF        interpolating shell.    -   (5) The identical smoothness of the DAF wavelet space and DAF        state space underlie the inherent robustness of the DAF wavelet        nets.

REGULARIZED DAF WAVELET NETS

In general, signal filtering may be regarded as a special approximationproblem with noise suppression. According to DAF polynomial theory, asignal approximation in DAF space can be expressed as $\begin{matrix}{{\hat{g}(x)} = {\sum\limits_{i}{{g( x_{i} )}\quad{\delta_{\alpha}( {x - x_{i}} )}}}} & (278)\end{matrix}$where the δ_(a)(x) is a generalized symmetric Delta functional. Wechoose it as a Gauss modulated interpolating shell, or the so-calldistributed approximating functional (DAF) wavelet. The Hermite-type DAFwavelet is given in the following equation [10]. $\begin{matrix}{{\delta_{M}( {x❘\sigma} )} = {{\frac{1}{\sigma}\quad\exp} - {( \frac{- x^{2}}{2\quad\sigma^{2}} ){\sum\limits_{n = 0}^{M/2}{( {- \frac{1}{4}} )^{n}\quad\frac{1}{\sqrt{2\quad\pi}{n!}}{H_{2n}( \frac{x}{\sqrt{2}\sigma} )}}}}}} & (279)\end{matrix}$

The function H_(2n) is the Hermite polynomial of even order 2n. Thequalitative behavior of one particular Hermite DAF is shown in FIG. 84.

The DAF wavelet neural nets possess the alternative feature of thecommonly used DAF approximation as $\begin{matrix}{{\hat{g}(x)} = {\sum\limits_{i}\quad{{w(i)}\quad{\delta_{a}( {x - x_{i}} )}}}} & (280)\end{matrix}$

The weights w(i) of the nets determine the superposition approximationĝ(x) to the original signal g(x)∈L²(R). It is easy to show that theweights of the approximation nets, w(i), are closely related to the DAFsampling coefficients g(x_(i)). The irregular finite discrete timesamplers of the original signal are selected for network learning. Ifthe observed signal is limited to an interval I containing a total of Ndiscrete samples, I={0, 1, . . . , N-1}, the square error of the signalis digitized according to $\begin{matrix}{E_{A} = {\sum\limits_{n = 0}^{N - 1}\quad{\quad\lbrack {{g(n)} - {\hat{g}(n)}} \rbrack^{2}}}} & (281)\end{matrix}$

This cost function is commonly used for neural network training in anoise-free background and is referred to as the minimum mean squareerror (MMSE) rule.

However, if the observed signal is corrupted by the noise, the networkproduced by MMSE training causes an unstable reconstruction, because theMMSE recovers the noise components as well as the signal. In this case,the signal-noise-ratio (SNR) cannot be improved much. Even for anoise-free signal, MMSE may lead to Gibbs-like undulations in thesignal, which is harmful for calculating accurate derivatives. Thus, formore robust filtering, the network structure should be modified to dealwith the particular situation. In this paper, we present a novelregularization design of the cost function for network training. Itgenerates edge-preserved filters and reduces distortion. To define theregularity (smoothness) of a signal, we introduce a “Lipschitz index”[6].Definition 1: Let f(x)∈L²(R), for any α>0, then if|f(x)-f(y)|=O(|x-y|^(α))  (282)the signal f(x) is said to be unified Lipschitz in the space L²(R). Theconstant, α, is the Lipschitz index of f(x).

It is easy to show that when the Lipschitz index, α, is an integer, theLipschitz regularity is equivalent to the differentiability of f(x) withsame order. For commonly used signals, the Lipschitz index α>0. In thepresence of noise distortion; the Lipschitz index always satisfies α<−1.This is because the noise causes sudden phase changes at neighboringpoints. To eliminate non-ideal undulations, we need to preserve thesignal trends while making a small MSE approximation. To achieve this,an additional smooth derivative term, E_(r), is introduced to modify theoriginal cost function. The new cost function is then$\quad\begin{matrix}{E = {{E_{A} + {\lambda\quad E_{r}}} = {\sum\limits_{k}\quad{\quad{\lbrack {{g(k)} - {\hat{g}(k)}} \rbrack^{2} + {\lambda{\int_{R}{\quad{\lbrack \frac{\partial{{\,^{r}\hat{g}}(x)}}{\partial x^{r}}\quad \rbrack^{2}{\mathbb{d}x}}}}}}}}}} & (283)\end{matrix}$

The factor l introduces a compromise between the orders of approximationand smoothness. Generally, the derivative order r≧2 is used to evaluatethe smoothness of the signals. Using the properties of the Hermite DAFs,we find that the derivative term of the regularized cost function E_(r)can be expressed in a comparatively simple convolution form,$\begin{matrix}{\frac{\partial{{\,^{r}\hat{g}}(x)}}{\partial x^{r}}\quad = {\sum\limits_{i}\quad{{w(i)}\quad\delta_{M}^{(r)}\quad( {{x - x_{i}}❘\sigma} )}}} & (284)\end{matrix}$where λ is termed a “differentiating DAF” and is given by$\begin{matrix}{{\delta_{M}^{(r)}\quad( {{x - x_{i}}❘\sigma} )} = {\frac{( {- 1} )^{r}}{2^{r/2}\sigma^{r + 1}}\quad{\exp( \frac{- ( {x - x_{i}} )^{2}}{2\quad\sigma^{2}}\quad )} \times {\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\quad\frac{1}{\sqrt{{2\quad\pi}\quad}{n!}}{H_{{2\quad n} + r}( \frac{x - x_{i}}{\sqrt{2\quad\sigma}} )}}}}} & (285)\end{matrix}$

It is exactly the rth derivative of δσ. Because of the infinitesmoothness of DAF wavelets, any order derivative can be obtained. TheHermite polynomial H is generated by the usual recursion $\begin{matrix}{{H_{n}(x)} = \{ \begin{matrix}{1,} & {n = 0} \\{{2\quad x},} & {n = 1} \\{{{2\quad x\quad{H_{n - 1}(x)}} - {2\quad( {n - 1} )\quad{H_{n - 2}(x)}}},} & {n > 1}\end{matrix} } & (286)\end{matrix}$

The predominant advantage of the Hermite polynomial approximation is itshigh-order derivative preservation (which leads to a smoothapproximation).

To increase the stability of the approximation system further, anadditional constraint in state space is taken to be $\begin{matrix}{E_{W} = \frac{\sum\limits_{i}\quad{{w(i)}}^{2}}{\sum\limits_{i}\quad{{g( x_{i} )}}^{2}}} & (287)\end{matrix}$

Thus the complete cost function utilized for DAF wavelet net training isgiven by $\begin{matrix}{E = {{E_{A} + {\lambda\quad E_{r}} + {\eta\quad E_{W}}} = {\sum\limits_{k}\quad{\quad{\lbrack {{g(k)} - {\hat{g}(k)}} \rbrack^{2} + {\lambda{\int_{R}{\quad{{\lbrack \frac{\partial{{\,^{r}\hat{g}}(x)}}{\partial x^{r}}\quad \rbrack^{2}{\mathbb{d}x}} + {\eta\quad\frac{\sum\limits_{i}\quad{{w(i)}}^{2}}{\sum\limits_{i}\quad{{g( x_{i} )}}}}}}}}}}}}} & (288)\end{matrix}$

SIMULATIONS

Two biomedical signal processing applications (for electrocardiogram andelectromyography) using the DAF wavelet neural nets are presented inthis chapter.

Automatic diagnosis of electrocardiogram (ECG or EKG) signals is animportant biomedical analysis tool. The diagnosis is based on thedetection of abnormalities in an ECG signal. ECG signal processing is acrucial step for obtaining a noise-free signal and for improvingdiagnostic accuracy. A typical raw ECG signal is given in FIG. 85. Theletters P, Q, R, S, T and U label the medically interesting features.For example, in the normal sinus rhythm of a 12-lead ECG, a QRS peakfollows each P wave. Normal P waves rate 60-100 bpm with <10%variations. Their heights are <2.5mm and widths <0.11 s in lead II. Anormal PR interval ranges from 0.12 to 0.20s (3-5 small squares). Anormal QRS complex has a duration of <0.1 2s (3 small squares). Acorrected QT interval (QTc) is obtained by dividing the QT interval withthe square root of the preceding R—R′ interval (normally QTc=0.42s). Anormal ST segment indicates no elevation or depression. Hyperkalaemia,hyperacute myocardial infarction and left bundle can cause an extra tallT wave. Small, flattened or inverted T waves are usually caused byischaemia, age, race, hyperventilation, anxiety, drinking iced water,LVH, drugs (e.g. digoxin), pericarditis, PE, intraventricular conductiondelay (e.g. RBBB) and electrolyte disturbance [12].

An important task of ECG signal filtering is to preserve the truemagnitudes of the P, Q, R, S, T, and U waves, protect the true intervals(starting and ending points) and segments, and suppress distortionsinduced by noise. The most common noise in an ECG signal is ACinterference (about 50 Hz-60 Hz in the frequency regime). Traditionalfiltering methods (low-pass, and band-elimination filters, etc.)encounter difficulties in dealing with the AC noise because the signaland the noise overlap the same band. As a consequence, experienceddoctors are required to carry out time-consuming manual diagnoses.

Based on a time varying processing principle, a non-linear filter [4]was recently adopted for ECG signal de-noising. Similar to the selectiveaveraging schemes used in image processing, the ECG is divided intodifferent time segments. A sample point classified as “signal” issmoothed by using short window averaging, while a “noise” point istreated by using long window averaging. Window width is chosen accordingto the statistical mean and variance of each segment. However, thiscalculation is complicated and it is not easy to select windows withappropriate lengths. The regularized spline network and wavelet packetswere later used for adaptive ECG filtering [5, 6], which is not yetefficient and robust for signal processing. In our present treatment,regularized DAF networks are used to handle a real-valued ECG signal. Weutilize our combined group-constraint technique to enhance signalcomponents and suppress noise in successive time-varying tilings.

The raw ECG of a patient is given in FIG. 86(a). Note that it hastypical thorn-like electromagnetic interference. FIG. 86(b) is theresult of a low-pass filter smnoothing. The magnitudes of the P and Rwaves are significantly reduced and the Q and S waves almost disappearcompletely. The T wave is enlarged, which leads to an increase in the QTinterval. Notably, the ST segment is depressed. Such a low-passfiltering result can cause significant diagnostic errors. FIG. 3(c) isobtained by using our DAF wavelet neural nets. Obviously, our methodprovides better feature-preserving filtering for ECG signal processing.

Another application is for electromyography (EMG) filtering. Surface EMGhas been used to evaluate muscle activation patterns in patients withgait disorders since the mid 1900s. In experimental as well as routinerecording of muscle action potentials, signal cross-talk from varioussources cannot always be avoided [2]. In particular EMG-investigationswithin the areas of physical science, orthopedics or ergonomics, wherethe collection of data has to be carried out under field conditions, themeasured signals are often incorrect due to movement of the subject. Inparticular DC off-set-voltages, movement of electrodes and cables, 50 Hzinterference and electrostatic interference should all be considered.But even with the utmost case, movement artifacts, particularly instudies of movement, cannot be completely avoided. Thus for a number ofquantitative signal processing procedures, an elimination ofinterference has to be carried out.

Once the raw scanning EMG data are stored in the computer, severalprocessing options are available for improving the signal quality. Thefirst one is removing the DC offset per recorded trace for eliminationof movement artifacts. The second option is a moving window smoothing(three points or more) in the time-direction of each trace for noisereduction. The third, a nonlinear median filtering over 3, 5 or 7 pointsis employed in the depth direction for elimination of non-time-lockedactivity. The iteration times of median filtering are performed,depending on the quality of the recorded scan [2]. Although a medianfilter is better for impulse-like noise removal than linear filters(low-pass, high-pass or band-limited) [11], it is not optimal since itis typically implemented uniformly across the image. The median filtersuppresses both the noise and the true signal in many applications. Itresults in amplitude reduction of the sharp signal peak, which isdeleterious for diagnostic analysis.

In this paper, we use a DAF wavelet neural network for the adaptive EMGprocessing. The sampling is irregular to match the time-varyingcharacteristics of EMG. Additional regularization and robust designsenable the optimal smooth approximation of the signal. As shown in FIG.87, the original measured EMG has many thorn-like noise peaks (FIG.87(a)). A simple low-pass filtered result is shown in FIG. 86(b). Ournewly developed technique results in the solution shown in FIG. 87(c).Again, one should note the presentation of feature details achieved.

CONCLUSION

Regularized DAF wavelet neural networks are proposed for non-stationarybiomedical signal processing. The DAF approximation shells possessinfinite smoothness in both physical and frequency domains, which enablethe high-resolution time-varying analysis of the signal. The optimalsignal filtering solution is obtained using a combination of severaldifferent contributions to a “cost function”. Measured ECG and EMGsignals are employed for testing the new technique. The simulations showthat our method is both efficient and robust for time-varying filtering.

REFERENCES

-   -   [1] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.        Information Theory, vol. 41, no.3, pp. 613˜627, 1995.    -   [2] T. H. J. M. Gootzen, “Muscle Fibre and Motor Unit Action        Potentials,” Ph.D. Thesis, Univ. of Nijmegen, 1990.    -   [3] D. K. Hoffmnan, G. W. Wei, D. S. Zhang, D. J. Kouri,        “Shannon-Gabor wavelet distributed approximating functional,”        Chemical Pyiscs Letters, Vol. 287, pp.119-124, 1998.    -   [4] Z. Shi, “Nonlinear processing of ECG signal,” B.Sc. Thesis,        Xidian Univ., 1991.    -   [5] Z. Shi, Z. Bao, L. C. Jiao, “Nonlinear ECG filtering by        group normalized wavelet packets”, IEEE International Symposium        on Information Theory, Ulm, Germany, 1997    -   [6] Zhuoer Shi, L. C. Jiao, Z. Bao, “Regularized spline        networks,” IJCNN'95, Beijing, China. (also J. Xidian Univ.,        Vol.22, No.5, pp.78˜86, 1995.)    -   [7] M. Unser, A. Aldroubi, and M. Eden, “B-spline signal        processing: part I—theory,” IEEE Trans. SP, Vol. 41, No.2,        pp.821˜833, February 1993    -   [8] M. Unser, A. Aldroubi, and M. Eden, “B-spline signal        processing: part II—Efficient design and applications,” IEEE        Trans. SP, Vol. 41, No.2, pp.834˜848, February 1993    -   [9] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,        “Lagrange distributed approximating Functionals,” Phys. Rev.        Lett., Vol. 79, No. 5, pp. 775˜779, 1997    -   [10] D. S. Zhang, G. W. Wei, D. J. Kouri, and D.K. Hoffman,        “Numerical method for the nonlinear Fokker-Planck equation,”        Phys. Rev. E, Vol.56, No. 1, pp.1197-1206, 1997.    -   [11] V. R. Zschorlich, “Digital filtering of EMG-signals,”        Electromyogr. Clin. Neurophysiol., Vol. 29, pp. 81-86, 1989.

PERCEPTUAL NORMALIZED SUBBAND IMAGE RESTORATION Introduction

An earlier group normalization (GN) technique [1] is extended to accountfor human perceptual responses. The purpose of the GN is to re-scale themagnitudes of various subband filters and obtain normalized equivalentdecomposition filters (EDFs). Our human visual system can be regarded asa natural signal and image processing devise—a non-ideal band passfilter. In order to achieve the best noise-removing efficiency, humanvision response [2] is accounted for by a perceptual normalization (PN).These approaches are combined with our biorthogonal interpolatingfilters [3] to achieve excellent perceptual image restoration andde-noising.

The biorthogonal interpolating filters are constructed by Swelden'slifting scheme [4] using Gaussian-Lagrange distributed approximatingfunctionals (DAFs) [5] which were introduced as an efficient andpowerful grid method for numerically solving partial differentialequations with extremely high accuracy and computational efficiency.From the point of view of wavelet analysis, GLDAFs can be regarded assmooth low pass filters.

PERCEPTUAL NORMALIZATION

The main objective of wavelet analyses is to provide an efficient L²(R)representation of a function. For signal and image processing, it isimportant to preserve the signal components of the corresponding subbandfilters. Wavelet coefficients can be regarded as the output of thesignal passing through the equivalent decomposition filters (EDFs).

The responses of the EDFs are the combination of several recurrentsubband filters at different stages. Wavelet coefficients can beregarded as the output of the signal passing through the equivalentdecomposition filters (EDF). The responses of the EDF are thecombination of several recurrent subband filters at different stages. Asshown in FIG. 88, the EDF amplitudes of our DAF-wavelets in eachsub-blocks are different. Thus the magnitude of the decompositioncoefficients in each of the sub-blocks cannot exactly reproduce theactual strength of the signal components. To adjust the magnitude of theresponse in each block, the decomposition coefficients are re-scaledwith respect a common magnitude standard. Thus EDF coefficients,C_(m)(k), in block m should be multiplied with a magnitude scalingfactor, λ_(m), to obtain an adjusted magnitude representation [1]. Thisidea was recently extended to Group Normalization (GN) of waveletpackets for signal processing [1].

The main objective of wavelet signal filtering is to preserve importantsignal components, and efficiently reduce noisy components. To achievethis goal, we utilize the human vision response to different frequencybands. An image can be regarded as the impulse response of the humanvisual system. The latter essentially consists of a series of “subbandfilters”. It is interesting to note that, just like the non-uniformresponse of wavelet subband EDFs, the human “subband filters” also havenon-uniform responses to a real object. Using a just-noticeableperceptual matrix, we can efficiently remove the visual redundancy fromdecomposition coefficients and normalize them with respect to thestandard of perception importance. A mathematical model for perceptionefficiency has presented by A. B. Watson, etc. [2] and can be used toconstruct the “perceptual lossless” quantization matrix Y_(m) fornormalizing visual response [2]. This treatment provides a simplehuman-vision-based threshold technique [6] for the restoration of themost important perceptual information in an image. For gray-scale imageprocessing, the luminance (magnitude) of the image pixels is what we aremost concerned with. The combination of the above mentioned group andvisual response normalization can be called the Visual GroupNormalization (VGN) of wavelet coefficients.

BIORTHOGONAL INTERPOLATING DAF WAVELETS

One of most important aspects of wavelet transforms is the waveletsthemselves. From the point of view of applications, the stability,regularity, time-frequency localization and computational efficiency arethe most important criteria for selecting wavelets. Interpolatingwavelets are particularly efficient for basis set construction sincetheir multiresolution spaces are identical to the discrete samplingspaces. In other words, there is no need for one to compute waveletexpansion coefficients by the usual inner products. This makes it easyto generate subband filters in a biorthogonal setting without requiringtedious iterations. Moreover, adaptive boundary treatments andnon-uniform samplings can be easily implemented using interpolatingmethods. The lifting scheme discussed by Swelden [4] is used in thiswork for constructing interpolating filters.

We use the interpolating Gaussian-Lagrange DAF (GLDAF) [5]${\phi_{M}(x)} = {{{W_{\sigma}(x)}{P_{M}(x)}} = {{W_{\sigma}(x)}\quad{\prod\limits_{{i = {- M}},{i \neq 0}}^{M}\frac{x - i}{- i}}}}$as a scale function for our wavelet construction. Here W_(σ)(x) is awindow function and is selected as a Gaussian because it satisfies theminimum frame bound condition in quantum physics. The quantity σ is awindow width parameter, and P_(M)(x) is the usual Lagrange interpolationkernel. Biorthogonal dual DAF scaling functions and DAF-wavelets areconstructed using Swelden's lifting scheme [4]. The present DAF-waveletsare extremely smooth and rapidly decay in both time and frequencydomains. As a consequence, they are free of Gibbs oscillations, whichplague most conventional wavelet bases. As plotted in FIG. 89, ourscaling functions and wavelets display excellent smoothness and rapiddecay.

DEMONSTRATIONS

We use as benchmarks the 512×512 Y-component Lena and Barbara images todemonstrate the restoration efficiency of the present approach. Theso-called “Lena” image possesses clear sharp edges, strong contrast andbrightness. The high texture component and consequently high frequencyedges in the Barbara image create considerable difficulties for manycommonly used filtering techniques. A simple low-pass filter will notonly smooth out the high frequency noise but also blur the image edges,while a simple high-pass filter can preserve the texture edges but willalso cause additional distortion.

As shown in FIG. 49(a) and FIG. 50(a), the original Lena and Barbaraimages are each degraded by adding Gaussian random noise. The medianfiltering (with a 3×3 window) result is shown in FIG. 49(b) and FIG.50(b), which is edge-blurred with low visual quality. It is evident thatour perceptual DAF wavelet technique (FIG. 49(c) and FIG. 50(c)) yieldsbetter edge-preservation and provides high visual restorationperformance.

REFERENCES

-   -   [1] Zhuoer Shi, Z. Bao, “Group-normalized wavelet packet signal        processing”, Wavelet Application IV, SPIE, Vol. 3078,        pp.226˜239, 1997    -   [2] A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor        “Visibility of wavelet quantization noise,” IEEE Trans. Image        Proc., vol. 6, pp. 1164-1175, 1997.    -   [3] Zhuoer Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman,        “Perceptual Image Processing Using Gaussian-Lagrange Distributed        Approximating Functional Wavelets,” submitted to IEEE SP        Letters, 1998.    -   [4] W. Swelden, “The lifting scheme: a custom-design        construction of biorthogonal wavelets,” Appl. And Comput.        Harmonic Anal., vol.3, no.2, pp.186˜200, 1996    -   [5] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,        “Lagrange distributed approximating Functionals,” Physical        Review Letters, Vol.79, No.5, pp. 775˜779, 1997    -   [6] D. L. Donoho, “De-noising by soft-threshold,” IEEE Trans.        Information Theory, vol. 41, no.3, pp. 613˜627, 1995

QUINCUNX INTERPOLATING 2D AND 3D WAVELETS-DAF Introduction

Nowadays, wavelets has became an important area of research for imageprocessing; however, most of the work done until today, it has beenconcentrated for the one dimensional case while the multi-dimensionalcase has been approached using the tensor product or known as well asseparable wavelet transform. In this work we discuss a way forconstructing “non-separable” wavelets in two and three dimensions usingdistributed approximating functionals (known as DAF) . A recent workpublished in 1998 proposes a way of constructing wavelets in onedimension called wavelets-DAFs. This work proves that this kind ofwavelets improves the accuracy for solving linear and non-linear partialdifferential equations. Using a non-separable sampling known as Quincunxfor the 2D case and FCO for the 3D case (or Quincunx in 3D) allows us tohave a better perception for the human visual system. The first sectionof this disclosure will explain the Quincunx sampling in 2D and 3D(FCO). Then, the second section addresses the construction of thescaling function. The third section explains the way of getting theactual values for the interpolating multidimensional filter.

NONSEPARABLE LATTICES

The quincunx lattice is considered as the simplest way of sampling intwo dimensions. We need in the 2D case two vectors to define thesampling, one possible value for those vectors is${d_{1} = {{\begin{pmatrix}1 \\1\end{pmatrix}\quad{and}\quad d_{2}} = \begin{pmatrix}1 \\1\end{pmatrix}}}\quad$which lead to the following dilation matrix $\begin{matrix}{D = ( \quad\begin{matrix}1 & 1 \\1 & {- 1}\end{matrix}\quad )} & (289)\end{matrix}$

In the Quincunx lattice, the subsampling process rotates the image by45° and flips the image about the horizontal axis.

As you can see in FIG. 90, the quincunx lattice has a checkerboardpattern. FCO lattic is used for the 3D case. The FCO lattice belongsalso to the general checkerboard lattices. One of the possible dilationmatrices for the FCO is $\begin{matrix}{D = ( \quad\begin{matrix}1 & 0 & 1 \\1 & 1 & 0 \\0 & 1 & 1\end{matrix}\quad )} & (290)\end{matrix}$notice that det(D)=M=2, for either quincunx or FCO.Scaling Function

Due to the fact that the scaling function behaves as generalized deltasequences we use the DAF approach to generate it. The two most importantaspects in the design of the scaling function are the decay rate andsmoothness. We studied the following functions and their properties.First, the normal function in 2D defined as follows $\begin{matrix}{{N( {x,y} )} = {\mathbb{e}}^{{{- {{x^{2} + y^{2}}}^{2}}/2}\sigma^{2}}} & (291)\end{matrix}$which obviously has a Gaussian decay, and its sigma defines the width ofthe neighborhood points. Another, function considered because of itsproperties under the Fourier transforms is the sinc equation defined asfollows $\begin{matrix}{{\sin\quad c\quad{\alpha( {x^{2} + y^{2}} )}} = \frac{\sin\quad\alpha\quad{\pi( {x^{2} + y^{2}} )}}{\alpha\quad{\pi( {x^{2} + y^{2}} )}}} & (292)\end{matrix}$

The combination of these functions produce the father wavelets which areboth smooth and rapidly decaying $\begin{matrix}{{\phi( {x,y} )} = {\frac{\sin\quad\alpha\quad{\pi( {x^{2} + y^{2}} )}}{\quad{\alpha\quad{\pi( {x^{2} + y^{2}} )}}}{\mathbb{e}}^{{{- {({x^{2} + y^{2}})}^{2}}/2}\sigma^{2}}}} & (293)\end{matrix}$

The next section explain the domain of x and y for equation 293. FIG. 91shows the scaling function for the quincunx lattice when α=σ=1. Equation(293), it is clearly a nonseparable function.

CONSTRUCTION OF THE QUINCUNX INTERPOLATING FILTER

We want to interpolate the red dot shown on FIG. 92, which coordinatesare (½, ½). The plane xy is the one that we obtain after we downsamplethe original lattice using quincunx. Then, this xy plane was rotated by45° with respect to the original lattice, and it was flipped about thehorizontal axis.

The circles in FIG. 92 represent, the values known from function. Thosevalues will be used to approximate the value of the function at the redpoint (½,½) called P. Those points are labeled depending on the distancewith respect to P. For instance, all points labeled “1” have the samedistance to P, (2)^(−(½)). Those points represent the filter taps. Therehas been some work on calculating the filter taps for the quincunx in 2Dand 3D, already; however, those filter taps are fix for the point at Pat (½,½). In our work, we recalculate the filters depending on where inthe grid is your new point P. The know values shown on FIG. 92 are fix,but not their labels. For instance, if our point P is located at (⅓,⅓)the values of the filter taps in the grid will change as shown on FIG.83.

This is something different with respect other works in this area. Wecalculate the value of those filter taps using Equation 293, where x andy are the difference between the points on the grid and the point P. Thesummation of all the values of the filter taps must be 1. In the casewhen we have only four points to approximate P, when P is located at(½,½) the result is trivial since we have four taps labeledone;therefore they must have the same value. Since their summation mustbe equal to one we resolve that those taps T(1)=¼. However, when we havemore points to approximate P, this is not that trivial. Using the DAFtheory we have to look for the optimal sigma and alpha to estimate thosefilter taps. We could even consider different values of sigma dependingon which points on the grid we are using to predict the value of thefunction at P.

The selection of the scaling function was crucial, thinking aboutcomputational time. Moreover, since we use DAFs to calculate the filterstaps Equation 293allows us to interpolate the n-derivate of the functionat point P.

DAF APPROACH TO IMAGE RFSTORATION Introduction

Anisotropic diffusion using nonlinear partial differential equations(PDEs) was proposed by Perona and Malik [1] for removal of noise, edgedetection, and image enhancement. Considerable theoretical and numericalanalyses and improvements have been carried out since its proposal[1-12]. In [1], the restored image is obtained by evolving the originalnoisy image u₀(x,y) according to the following diffusion equation with avariable diffusion coefficient $\begin{matrix}{\frac{\partial{u( {x,y,t} )}}{\partial t} = {{div}\lbrack {{c( {{\nabla{u( {x,y,t} )}}} )}{\nabla{u( {x,y,t} )}}} \rbrack}} & (294)\end{matrix}$where, the initial condition is u(x,y,0)=u₀(x,y), where div defines thedivergence operator, and c(|∇u|) is a nonincreasing positive diffusioncoefficient function. It is interest to see that when c is a constant C,Equation (294) reduces to the heat conduction or isotropic diffusionequation, as follows $\begin{matrix}{\frac{\partial{u( {x,y,t} )}}{\partial t} = {C{\nabla^{2}{u( {x,y,t} )}}}} & (295)\end{matrix}$

This equation has been used to identify objects in image processing[13]. However, using the isotropic diffusion for image restorationresults in a severe edge-blurring problem, which is the principal reasonfor introducing anisotropic diffusion. The basic idea of anisotropicdiffusion is: [1] for a region with small gradient |∇u|, the diffusioncoefficient c will be large to encourage smoothing this region; for alarge |∇u| region, c will be small to discourage the diffusion for thepurpose of preserving the edges. In [1], Perona and Malik suggested thefollowing two expressions for the diffusion coefficient, $\begin{matrix}{{c( {{\nabla u}} )} = {\exp\lbrack {- ( \frac{{\nabla u}}{k} )^{2}} \rbrack}} & (296)\end{matrix}$and $\begin{matrix}{{{c( {{\nabla u}} )} = \frac{1}{1 + ( \frac{{\nabla u}}{k} )^{2}}},} & (297)\end{matrix}$where k is a constant used to provide a threshold for noise removal,i.e., to enhance the edges and smooth the homogeneous areas. However, aspointed out by many authors [2,4,6,8], such a procedure Equation (294)with diffusion coefficients given by Equation (296) or Equation (297) isill posed. When the image is corrupted with Gaussian noise, it can occurthat the gradient produced by noise is comparable to that produced bythe edges. Application of the anisotropic diffusion equation to such animage will tend to preserve both the image edges and the undesirablenoisy edges simultaneous. Nevertheless, in order for the solution of thediffusion equation to be unique, |∇u|c(|∇u|) must be nondecreasing.Otherwise, the solution may diverge in time for some choices ofnumerical schemes. The same image with a minor difference amount ofnoise may lead much different solutions using the same numerical scheme.The solutions of different numerical schemes to the same image with thesame amount of noise may also have much difference. To alleviate theill-posed problem, it is crucial to detect the edges as accurately aspossible, but the edges of a image are usually unknown to us. For thispurpose, a selective smoothing of the diffusion coefficient in Equation(294) was introduced in [2], as follows: $\begin{matrix}{{\frac{\partial{u( {x,y,t} )}}{\partial t} = {{div}\{ {{c\lbrack {{\nabla( {{g( {x,y} )}*{u( {x,y,t} )}} )}} \rbrack} \times {\nabla{u( {x,y,t} )}}} \}}},} & (298)\end{matrix}$where g(x,y) is a Gaussian filter to detect the edges andg(x,y)*f(x,y,t) denotes the convolution operation at a given time t.This smoothing operation is not only limited to the use of a theGaussian filter. In [9], a symmetric exponential filter was proposed,and was claimed to be more accurate for detection the position of edges.It is given byg(x, y)={fraction (β/2)}exp (−β(|x|+|y|))  (299)

Using this smoothing filter, significant improvement in the quality ofthe restored images was attained [9]. However, Equation (298) has aserious drawback of requiring compution of the convolution because it isrequired at each time step [8].

We disclose using a recently developed distributed approximatingfunctional (DAF) approach [15-17] to solve the anisotropic diffusionproblem. The DAFs used here have several advantages. First, they can beused to spatially discretize the diffusion equation. Previousapplications of the DAFs to nonlinear partial differential equations(PDEs) has shown that they are very accurate and stable PDE solvers[18-19]. Second, by proper choice of the parameters, the DAF evaluationof |∇f| in the diffusion coefficient automatically produces smoothing.Third, a periodic boundary condition is added to the image, using DAFsaccording to a recently developed algorithm [20,22], making it possibleto utilize DAFs and many other numerical schemes which require aknowledge of the boundary conditions. Fourth, the DAF can be used as anapproximate ideal low pass filter to remove the high frequency noisebefore performing the time evolution. The success of DAFs applied to theanisotropic diffusion equation is due to their so called “well-tempered”property, which is the most important feature in the DAF theory [17].

DISTRIBUTED APPROXIMATING FUNCTIONAL FORMALISM

The distributed approximating functionals (DAFs) were introduced as“approximate identity kernels” used to approximate a function and itsvarious derivatives, in terms of a discrete sampling [15-17]. One of themost important properties of one class of commonly used DAFs is theso-called “well-tempered” approximation, which distinguishes them frommany other numerical schemes. The “well-tempered” DAFs are constructedsuch that there are no “special points” in the DAF approximation; i.e.,the approximation to a function has a similar order of accuracy both onand off the input grid points. The DAFs also provide a “well-tempered”approximation to the derivatives. As long as the function and itsderivatives remain in the “DAF-class”, the “well-tempered” property willensure that they have similar accuracy. In contrast, many othernumerical schemes (e.g., basis expansions, wavelets, splines, finitedifferences, finite elements, etc.), yield exact results for thefunction on the grid points, but typically at the expense of givingsignificant worse results for the function off the grid points. Suchinterpolative methods are always less accurate for the derivatives ofthe function. The drawback of the interpolative methods is especiallydistinct when the function is contaminated by noise, since the noise inthe signal input data will be kept unchanged. In this case, a filtermust be used to eliminate the noise in advance. Another feature ofcertain DAFs is that since they are approximate integral identitykernels, they yield an integral representation of differentialoperators. By proper choice of the parameters, the DAFs provide acontrollably accurate analytical representation of derivatives on andoff the grid points, which is crucial to their success in manyapplications. One of the most useful realizations of the “well-tempered”DAFs is the Hermite DAF, given by $\begin{matrix}{{\delta_{M}( z \middle| \sigma )} = {\frac{\exp( {- z^{2}} )}{\sqrt{2\pi}\sigma}{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{n!}{H_{2n}(z)}}}}} & (300)\end{matrix}$where z=(x—x)|(√{square root over (2)}σ), H_(2n)(z) is the usual (evenorder) Hermite polynomial, and σ and M are the DAF parameters. The HDAFis dominated by its Gaussian factor exp(−z²), which serves to controlthe effective width of the function. In the limit of either M→∞ or σ→0,the Hermite DAF becomes identical to the Dirac delta function. Afunction f(x) is approximated by the Hermite DAF according to$\begin{matrix}{{{f(x)} \approx {f_{DAF}(x)}} = {\int_{- \infty}^{\infty}{{\delta_{M}( {x - x^{\prime}} \middle| \sigma )}{f( x^{\prime} )}\quad{\mathbb{d}x^{\prime}}}}} & (301)\end{matrix}$

For function sampling on a discrete, uniform grid, the Hermite DAFapproximation to the function at any point x (on or off the grid point)is given by $\begin{matrix}{{f_{DAF}(x)} =  {{\Delta{\sum\limits_{j}\quad{\delta_{M}( x }}} - x_{j}} \middle| {\sigma \quad ){f( x_{j} )}} } & (302)\end{matrix}$where Δ is the uniform grid spacing. Only terms in the sum from gridpoints sufficiently close to x contribute significantly.

Approximations for various linear transformations of a function can alsobe generated using the Hermite DAF. Especially important cases arederivatives to various orders, given by $\begin{matrix}{{{f^{(l)}(x)} \approx {f_{DAF}^{(l)}(x)}} = {\int_{- \infty}^{\infty}{{\delta_{M}^{(l)}\quad( {x - x^{\prime}} \middle| \sigma )}{f( x^{\prime} )}{\mathbb{d}x^{\prime}}}}} & (303)\end{matrix}$where δ_(M) ^((l))(x—x|σ) is the 1th derivative of δ_(M)(x—x′|σ), and$\begin{matrix}{{\delta_{M}^{(l)}( z \middle| \sigma )} = {\frac{2^{{- l}/2}{\exp( {- z^{2}} )}}{\sqrt{2\pi}\sigma^{l + 1}}{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{( {- 1} )^{l}}{n!}\quad{H_{{2n} + l}(z)}}}}} & (304)\end{matrix}$

These expressions can be discretized by quadrature, $\begin{matrix}{{{f^{(l)}(x)} \approx {f_{DAF}^{(l)}(x)}} =  {{\Delta{\sum\limits_{j}\quad{\delta_{M}^{(l)}( x }}} - x_{j}} \middle| {\sigma \quad ){f( x_{j} )}} } & (305)\end{matrix}$

Equation (11) and (12) provide convenient methods for calculatingderivatives in solving nonlinear partial differential equations (NPDEs).

The HDAF approximation can be easily extended to a multi-dimensionalcase as a direct product. For two-dimension function f(x,y), one has$\begin{matrix}{{f_{DAF}( {x,y} )} = {\Delta_{x}\Delta_{y}{\sum\limits_{j_{1}}\quad{{\delta_{M_{x}}( {x - x_{j_{1}}} \middle| \sigma_{x} )} \times {\sum\limits_{j_{2}}\quad{{\delta_{M_{y}}( {y - y_{j_{2}}} \middle| \sigma_{y} )}{f( {x_{j_{1}},y_{j_{2}}} )}}}}}}} & (306)\end{matrix}$and $\begin{matrix}{ {{f_{DAF}^{({l + m})}( {x,y} )} = {\Delta_{x}\Delta_{y}{\sum\limits_{j_{1}}\quad{{\delta_{M_{x}}^{(l)}( {x - x_{j_{1}}} \middle| \sigma_{x} )} \times {\sum\limits_{j_{2}}\quad{\delta_{M_{y}}^{(m)}( {y - y_{j_{2}}} \sigma_{y}}}}}}} )\quad{f( \quad{x_{j_{1}},y_{j_{2}}} )}} & (307)\end{matrix}$where f_(DAF) ^((l+m))(x,y) denotes 1th partial derivative and mthpartial derivative of the function with respect to x and y respectively.

The Hermite DAF has the so-called “well-tempered” property [17], and itis a low pass filter. In FIG. 94, we show plots of Hermite DAFs,obtained using two different set of parameters, in FIG. 94(a) coordinatespace and FIG. 94(b) frequency space, respectively. Their first orderderivatives are plotted in FIG. 94(c), in coordinate space. The solidline is for σ=3.54, M=120 and the dashed line is for σ=2.36, M=30. Thesolid line is said to be in the less “well-tempered” or more“interpolative” regime, and the dashed line is in the more“well-tempered” (smoothing) regime. In contrast to the sinc functionsin(wx)/πx, with w=πΔ (the ideal interpolator) being the idealinterpolation and w<π/Δ being the low pass filter, the Hermite DAF ishighly banded in coordinate space due to the presence of the Gaussianfactor, which means that only a small number of values are required oneach side of the point x in Equation (294), as is also true for thederivatives of the Hermite DAF. This is clearly seen in FIG. 94(a), andFIG. 94(c). With proper choice of parameters, the Hermite DAF is anexcellent filter (see the dashed line in FIG. 94). For the case far awayfrom interpolation, the DAF approximation is not exact on grids (TheHermite DAF is not equal to 1 at the origin and not equal to 0 at nΔ,n≠0). The DAF approximation to the function depends not only on thefunction value at the grid point itself, but also on values at the gridpoints close to it. The approximation to a function has about the sameorder of accuracy both on and off the grid points. This “well-tempered”property of DAFs plays an important role in our periodic extensionalgorithm. Application of the “well-tempered” DAF to approximate thefunction and its derivatives is relatively insensitive to the presenceof noise. As will be seen in this disclosure, the “well-tempered”Hermite DAF serves as a very good smoothing filter to accurately detectthe image edges. Application of the “well-tempered” Hermite DAF to thevariable of the diffusion coefficient in the anisotropic diffusionEquation (294) avoids the need for an additional smoothing operation.The Hermite DAF will perform the discretization and the smoothingoperation on the function simultaneously. Since the “well-tempered”Hermite DAF is also a low pass filter, by proper choice of the DAFparameters, Equation (306) can be used to eliminate the high frequencynoise of a signal once the boundary condition is known. However, theboundary condition is usually unknown. In the following section, we willgive a brief review of how a periodic boundary condition is imposed,using the DAFs

DAF-BASED METHOD OF PERIODIC EXTENSION

For a signal of finite length, it is common requirement for manynumerical schemes that one know the boundary conditions of the signal inorder to avoid “aliasing”. Take the noncausal finite impulse responsefilters (FIRs) as an example. In order to get the desired filteringresults, the noncausal FIRs require a knowledge of the signal values inthe future, which are impossible to obtain in the advance. Also, thereare many numerical analysis algorithms for which it is computationallyefficient for the signal to be a given length, which is not usuallysatisfied in experimental results. In this section, we show how ouralgorithm is employed to impose a periodic boundary condition for thesignal. The algorithm of periodic extension has been discussed in detailin [20] and [22]. We will give a brief description on how it isimplemented.

The method is closely related to the DAF extrapolation algorithmpresented in [21]. Assume a limited length of a signal {f₁,f₂, . . .,f_(J-1)} is known, and we require it to be periodic with the period Kso that K—J+1 values {f_(J),f_(J+1), . . . , f_(K)} are left to bedetermined. Since now the signal is periodic, the values{f_(K+1),f_(K+2), . . . ,f_(K+J—1)} are also known and equal to {f₁,f₂,. . . ,f_(J-1)} respectively. Under the assumption that the signal is inthe “DAF-class”, the unknown values from f_(j) to f_(K) can bedetermined by minimizing the following “cost function”, constructedusing the “well-tempered” Hermite DAF: $\begin{matrix}{C \equiv {\sum\limits_{p}{W_{p}( {{f( x_{p} )} - {f_{DAF}( x_{p} )}} )}^{2}}} & (308)\end{matrix}$where W_(p) is a weight related to the grid point x_(p) andf_(DAF)(x_(p)) is the DAF approximation to the function at x_(p). Thesummation range for “p” is selected according to the problem considered.From Equations (302) and (308), we have $\begin{matrix}{C \equiv {\sum\limits_{p}{\quad W_{p}( {{f( x_{p} )} - {\sum\limits_{t = {p - w}}^{p + w}\quad{\delta_{M}( x_{p} }} - {x_{t} \quad| {\sigma \quad ){f( x_{t} )}}  )^{2}}} }}} & (309)\end{matrix}$where w is the DAF half bandwidth. We minimize the “cost function” withrespect to the variation of the unknowns function according to$\begin{matrix}{{\frac{\partial C}{\partial{f( x_{1} )}} = 0},\quad{J \leq l \leq K},} & (310)\end{matrix}$

Equations (309) and (310) generate a set of linear simultaneousalgebraic equations, given by $\begin{matrix}{{{ {\sum\limits_{p = {l - w}}^{l + w}{\quad 2{W_{p}( {{f( x_{p} )} -}\quad }{\sum\limits_{t = {p - w}}^{p + w}{\quad{\delta_{M}( {x_{p} - x_{t}} \middle| \sigma )}{f( x_{t} )}}}}} ) \times \quad( {\delta_{pl} - {\delta_{M}( {x_{p} - x_{l}} \middle| \sigma )}} )} = 0},{J \leq l \leq K}} & (311)\end{matrix}$where δ_(pl) is the kronecker delta function. These linear algebraicequations can be solved by standard algorithms [18] to predict values ofthe function in the gap.

Note that it is the “well-tempered” property of the DAFs that makes theabove algorithm possible. For interpolation algorithms, the value oneach grid point is exact and does not dependent on the values of thefunction at any other grid points, which means that the “cost function”is always zero for points on the grid. In contrast, the “well-tempered”DAFs are not required to give a exact representation on the grid points.The approximate values on a grid point are related to the nearby values.

We must stress that this periodic extension of an nonperiodic signal isbasically different from extrapolation, although they use similaralgorithm. The “pseudo-signal” in the extended domain is only used as anaid to treat or an analyze the original signal in a way that doesn'tsignificantly disturb the behavior in the known region. In general, onedoes not obtain true signal in this domain. The beauty of the presentperiodic extension algorithm is that it provides a boundary condition ofthe signal without significant “aliasing”, as was shown in [20] and [22]Once the periodic signal is obtained, it can be extended as much asdesired, depending on the intended numerical application. In contrast,extrapolation is required to accurately predict the true function valuesin the extended domain.

It must be noted that although we have discussed the algorithm in onedimension, extending it to two or more dimensions is straightforward.The DAF approximation to a function and its derivatives are given inEquations (306) and (307), respectively. However, a directtwo-dimensional extrapolation is a time and memory consuming procedurebecause too many simultaneous algebraic equations must be solved. Onealternative to this is to consider the two dimensional patterns ascomposed of many lines of one-dimensional signals, and then extend eachline separately. This procedure is generally less accurate than thedirect two dimensional extrapolation algorithm because it only considersthe correlation along one direction while neglecting cross correlation.However, for many problems, it is sufficiently accurate, and it is avery economical procedure.

The algorithm presented in this section only refers to periodicextension as an example. However, we note that one can also extend theoriginal signal to any other kinds of boundary conditions, as needed fordifferent numerical applications. Periodic extension is only one caseout of many possible ones.

NUMERICAL EXAMPLES

Numerical experiments are preformed using the “Lena image”, shown inFIG. 95 and in Table 7. We degrade it with Gaussian white noise withpeak signal-to-noise ratio (PSNR) of 22.14 dB [FIG. 96(a)] and 18.76 dB[FIG. 97(a)] respectively.

TABLE 7 Comparative Restoration Results in PSNR for ImageLena Corruptedwith Different Amount of Gaussian Noise Corrupted image Algorithm 22.14dB 18.76 dB Median filter (3 × 3) 27.24 dB 24.72 dB Median filter (5 ×5) 26.53 dB 25.29 dB New approach 30.14 dB 28.19 dBThe PSNR used here is defined to be $\begin{matrix}{{PSNR} = \frac{255 \times 255}{MSE}} & (312)\end{matrix}$where MSE is the mean-square-error of the noisy image, defined by$\begin{matrix}{{MSE} = {\frac{1}{N_{x}N_{y}}{\sum\limits_{i = 0}^{N_{x} - 1}\quad{\sum\limits_{j = 0}^{N_{y} - 1}\quad\lbrack {{S( {i,j} )} - {\hat{S}( {i,j} )}} \rbrack^{2}}}}} & (313)\end{matrix}$where S(ij) and Ŝ(ij) are the original image and noisy image samplesrespectively, and N_(x) and N_(y) are the number of pixels horizontallyand vertically respectively. For the purpose of effectively removing thenoise in the homogeneous areas while simultaneously preserving theedges, we choose the following definition of the diffusion coefficient[8] $\begin{matrix}{{c(s)} = \{ \quad\begin{matrix}{{A\quad\frac{{p( {T + ɛ} )}^{p - 1}}{T}},} & {s < T} \\{{A\quad\frac{{p( {s + ɛ} )}^{p - 1}}{s}},} & {s \geq T}\end{matrix}\quad } & (314)\end{matrix}$where A, T, ∈ and p are adjustable parameters. Here we fix A=100, T=5,∈=1 and p=0.5 in our numerical applications. The anisotropic diffusionEquation (294) is spatially discretized using the Hermite DAFs accordingto Equation (307). The more “well-tempered” Hermite DAFs(σ_(x)/Δ_(x)=σ_(y)=2.36, M_(x)=M_(y)=30 for the lower noise image andM_(x)=M_(y)=22 for the higher noise image) are used to discretize thevariable of the diffusion coefficient to remove more noise andaccurately detect the edges. A more accurate Hermite DAF(σ_x/Δ_(x)=σ_(y)/Δ_(y)=3.54, M_(x)=M_(y)=120) is employed to discretizethe other part of the spatial differential operation accurately. Thefourth order Runge-Kutta method [23] is employed to perform the timeevolution. Before carrying out the time propagation, we used the HDAFwith σ_(x)=σ_(y)=2.36, M_(x)=M_(y)=32 to remove the high frequencynoise.

FIG. 96(b) and FIG. 97(b) show the resulting restored Lena images ofproduced by our numerical algorithm. Comparing them with the degradedimages [FIG. 96(a) and FIG. 97(a)] and the original image (FIG. 95), itis clearly seen that the algorithm can effectively remove noise whilepreserving the edges simultaneously. The PSNR is increased by about 8.00dB for the lower noise image and 9.43 dB for the higher noise image. Theboundary of the image is also effectively preserved, which shows thesuccess of our periodic extension algorithm. As pointed in [9], fornoisier images, the Perona-Malik algorithm may produce some noise echosthat are larger than the threshold level which can not be removed.However, increasing the threshold level will lead to the edge-blurringproblem. By using the algorithm presented in this disclosure, we caneffectively avoid the problem by accurately detecting the edges usingthe “well-tempered” HDAF. The procedure of using the “well-tempered”Hermite DAF to detect the edges automatically avoids having to apply anadditional smoothing operation to the variable of the diffusioncoefficient. We also test the success of our algorithm for even noiserimages and find that it can increase the PSNR enormously (e.g., we haveincreased the PSNR of the Gaussian noise degraded Lena image from 11.35dB to 23.10 dB). Computations with the smoothing filter Equation (299)applied to the variable of the diffusion coefficient, (as presented in[9]), shows that the “well-tempered” Hermite DAF is better able todetect and preserve the edges than the exponential filter.

CONCLUSIONS

New computational schemes are disclosed to restore a noise corruptedimage based on solving an anisotropic diffusion equation. Thedistributed approximating functional (DAF) approach introduced in thisdisclosure has several advantages for application to the anisotropicdiffusion equation. First, it provides a periodic boundary condition tothe image. Second, it provides a smoothing operation to the variable ofthe diffusion coefficient. Third, it discretizes the spatial derivativesof the equation as accurately as required for a given application. Inaddition, before performing the time evolution using the forth orderRunge-Kutta method, the DAF can be used to eliminate the high frequencynoise from the image. Test results on a noisy Lena image show that it iseffective for removing the noise and preserving the edges of the imagesimultaneously. It can greatly increase the PSNR of the noisy image. Theresulting periodic extended images can be implemented with many othernumerical schemes which require a knowledge of the signal in an extendeddomain or prefer a given number of image samples.

REFERENCES

-   -   [1] P. Perona and J. Malik, “Scale-space and edge detection        using anisotropic diffusion,” IEEE Trans. Pattern Anal. Machine        Intell., Vol. 12, pp. 629-639, 1990.    -   [2] F. Catte, P.-L Lions, J.-M. Morel, and T. Coll, “Image        selective smoothing and edge detection by nonlinear diffusion,”        SIAM J. Numer. Anal., Vol. 29, pp. 182-193, 1992.    -   [3] L. Alvarez, P.-L Lions, and J.-M Morel, “Image selective        smoothing and edge detection by nonlinear diffusion II,” SIAM J.        Numer. Anal., Vol. 29, pp. 845-866, 1992.    -   [4] M. Nitzberg and Shiota, “Nonlinear image filtering with edge        and corner enhancement,” IEEE Trans. Pattern Anal. Machine        Intell., Vol. 14, pp. 826-833, 1992.    -   [5] L. Rudin, S. Osher, and Emad, “Nonlinear total variation        based noise removal algorithm,” Physica D, Vol. 60, pp. 259-268,        1992.    -   [6] R. T. Whitaker and S. M. Pizer, “A multi-scale approach to        nonuniform diffusion,” CVGIP: Image Understanding, Vol. 57, pp.        99-110, 1993.    -   [7] Y.-L You, M. Kaveh, W.-Y Xu, and T. Tannenbaum, “Analysis        and design of anisotropic diffusion for image processing,” in        IEEE Proc. 1st Int. Conf. Image Processing, Austin, TX,        November, 1994, Vol. II, pp. 497-501.    -   [8] Y.-L. You, W. Xu, A. Tannenbaum, and M. Kaveh, “Behavior        analysis of anisotropic diffusion in image processing”, IEEE        Trans. Image Processing, Vol. 5, pp. 1539-1553, 1996.    -   [9] F. Torkamani-Azar, K. E. Tait, “Image recovery using the        anisotropic diffusion equation,” IEEE Trans. Image Processing,        Vol. 5, pp. 1573-1578, 1996.    -   [10] G. Sapiro, “From active contours to anisotropic diffusion:        connects between basic PDE's in image processing,” in IEEE Proc.        3rd Int. Conf. Image Processing, Lausanne, Switzerland,        September 1996, Vol. 1, pp. 477-480.    -   [11] J. Shah, “A common framework for curve evolution,        segmentation and anisotropic diffusion,” in IEEE Proc. Conf.        Computer Vision and Pattern Recognition, San Francisco, Calif.,        June 1996, pp. 136-142.    -   [12] S. T. Acton, “Edge enhancement of infrared imagery by way        of anisotropic diffusion pyramid,” in IEEE Proc. 3rd Int. Conf.        Image Processing}, Lausnne, Switzerland, September 1996, pp.        865-868.    -   [13] A. P. Witkin, “Scale-space filtering”, in Proc. Int. Joint        Conf. Artif. Intell., IJCAI, Karlsruhe, West Germany, 1983, pp.        1019-1021.    -   [14] S. Teboul, L. Blanc-Feraud, G. Aubert, and M. Barlaud,        “Variational approach for edge-preserving regularization using        coupled PDE's,” IEEE Trans. Image Processing, Vol. 7, pp.        387-397, 1998.    -   [15] D. K. Hoffman, M. Arnold, and D. J. Kouri, “Properties of        the optimum approximating function class propagator for        discretized and continuous wave packet propagations,” J. Phys.        Chem., Vol. 96, pp. 6539-6545, 1992.    -   [16] D. J. Kouri, W. Zhu, X. Ma, B. M. Pettitt, and D. K.        Hoffman, “Monte carlo evaluation of real-time Feynman path        integrals for quantal many-body dynamics: distributed        approximating functions and Gaussian sampling,” J. Phys. Chem.,        Vol. 96, pp. 9622-9630, 1992.    -   [17] D. K. Hoffman, T. L. Marchioro II, M. Arnold, Y. Huang, W.        Zhu, and D. J. Kouri, “Variational derivation and extensions of        distributed approximating functionals,” J. Math. Chem., Vol. 20,        pp. 117-140, 1996.    -   [18] D. S. Zhang, G. W. Wei, D. J. Kouri, and D. K. Hoffman,        “Numerical method for the nonlinear Fokker-Planck equation,”        Phys. Rev. E, Vol.56, pp.1197-1206, 1997.    -   [19] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman,        “Distributed approximating functional approach to Burgers'        equation in one and two space dimensions,” Comput. Phys.        Commun.}, Vol. 111, pp. 93-109, 1998.    -   [20] D. K. Hoffman, G. H. Gunaratne, D. S. Zhang, and D. J.        Kouri, “Fourier Filtering of Images,” Chaos, submitted.    -   [21] A. Frishman, D. K. Hoffman, R. J. Rakauskas, and D. J.        Kouri, “Distributed approximating functional approach to fitting        and predicting potential surfaces. 1. Atom-atom potentials,”        Chem. Phys. Lett., Vol. 252, pp. 62-70, 1996.    -   [22] D. S. Zhang, D. J. Kouri, D. K. Hoffman and G. H.        Gunaratne, “Distributed approximating functional treatment of        noisy signals”, submitted.    -   [23] W. H. Press, B. P. Flannery, S. A. Teukosky, and W. T.        Vetterling, Numerical Recipes—The Art of Scientific Computing,”        Cambridge Press, Cambridge, 1988.

A VARIATIONAL APPROACH TO THE DIRICHLET-GABOR WAVELET-DAF Introduction

Recently we have introduced several new types of distributedapproximating functionals (DAFs) [1-5], and related wavelet basesassociated with them [4,6-10]. In the course of that work, it wasobserved that DAFs could be generated in several ways. The firstapproach [2,3] leads to a systematic way of approximating a givendiscrete set of input data with an infinitely smooth function. The mostintensively studied DAF of this type is called the Hermite DAF, or HDAF.The fundamental unit of its construction is a product of a Hermitepolynomial and its generating function, referenced to an origin that islocated at each point, x. A variational method was introduced forderiving such DAFs [3], and they have been applied to a large number ofproblems, ranging from solving various linear and nonlinear partialdifferential equations (PDEs) to the entire gamut of signal processing[6,7,11-14]. A distinctive property of the first DAFs is that they arenot interpolative on the input grid points [2,3]. That is, the firsttype of DAF approximation to the function at any grid point, x_(j), isnot exactly equal to the input data value. In place of the interpolativeproperty, this DAF approach to functional approximation has the propertythat there are no “special points”. Said another way, such DAFs deliversimilar accuracy for approximating the function either on or off thegrid; similarly, the DAF approximation to a function, sampleddiscretely, yields an approximation to the derivatives of the functioncomparable in accuracy to the function itself. This is strictly trueonly for functions belonging to the “DAF-class”, which is that set offunctions whose Fourier transform is sufficiently contained under theDAF-window in Fourier space [3].

More recently, we have developed another general type of DAF which doesinterpolate on the grid, but which still can be “tuned” to yield highlyaccurate derivatives for DAF-class functions [4-7,11-14]. The essence ofthis approach is to modify an “interpolating shell” (such as that forLagrange Interpolation [5], etc.) by an appropriate weighting function.By far the most attractive choice has been a Gaussian weight function,which has the property of “regularizing”the interpolation so that itdelivers an infinitely smooth approximation to discretely sampledfunctions [4-7], and the accuracy is ensured so long as the functionbeing considered is in the DAF class. Again, these have been shown to beenormously robust for the class of PDEs and signal processing problemsconsidered earlier [5-7,11-14].

An alternative way of viewing these DAFs results from observing thatcontinuous DAFs constitute two-parameter Dirac delta sequences [4]. Thatis, they are approximate identity transforms that depend on twoadjustable parameters. In the case, e.g., of the HDAFs, the twoparameters are the Gaussian width, σ, and the highest degree polynomial,M (where M is even) $\begin{matrix}{{\delta_{DAF}^{(M)}( {x - x^{\prime}} \middle| \sigma )} = {{\frac{1}{\sigma}\exp} - {\lbrack \frac{- ( {x - x^{\prime}} )^{2}}{2\sigma^{2}} \rbrack{\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2\pi}{n!}}{H_{2n}( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )}}}}}} & (315)\end{matrix}$where, it is easily shown that $\begin{matrix}{{\lim\limits_{Marrow\infty}{\delta_{DAF}^{(M)}( {x - x^{\prime}} \middle| \sigma )}} = {\delta( {x - x^{\prime}} )}} & (316)\end{matrix}$for any σ>0, and also that $\begin{matrix}{{\lim\limits_{\sigmaarrow\infty}{\delta_{DAF}^{(M)}( {x - x^{\prime}} \middle| \sigma )}} = {\delta( {x - x^{\prime}} )}} & (317)\end{matrix}$for and fixed M.. The availability of two independent parameters, eitherof which can be used to generate the identity kernel or Dirac deltafunction, can be viewed as the source of robustness of the DAFs ascomputational tools [4].

Of the recently introduced regularized, interpolation DAFs, apotentially very useful one is the Dirichlet-Gabor wavelet-DAF (DGWD).It was constructed by combining a Gaussian with the Dirichlet kernel forgenerating the Fourier series of a function, to give $\begin{matrix}{{\delta_{DAF}^{(M)}( {x - x^{\prime}} \middle| \sigma )} = {C_{M,\sigma}{\mathbb{e}}^{{- \frac{{({x - x^{\prime}})}^{2}}{2\sigma^{2}}}\quad}\frac{\sin\lbrack {( {M + \frac{1}{2}} )\quad\frac{2\pi\quad x}{L}} \rbrack}{2{\sin( \frac{\pi\quad x}{L} )}}}} & (318)\end{matrix}$

As with all of the regularized, interpolating DAFs, this productgenerates a scaling wavelet that at once is infinitely smooth andrapidly decaying in both physical and Fourier space [4]. The constant,C_(M,σ), was determined by requiring that the zero frequency Fouriertransform, {circumflex over (δ)}^((M)) _(DGWD) (0|σ), be normalized tounity, that is $\begin{matrix}{{\hat{\phi}(0)} = {{{\hat{\delta}}_{DGWD}^{(M)}( 0 \middle| \sigma )} = {{\int_{- \infty}^{\infty}{{\mathbb{d}x}\quad{\delta_{DGWD}^{(M)}( x \middle| \sigma )}}} = 1}}} & (319)\end{matrix}$

Then the “father wavelet” basis is generated by translating and scaling,so that [4] $\begin{matrix}{{\phi_{mn}(x)} = {a^{\frac{- m}{2}}{\phi( \frac{x - {n\quad b}}{a^{m}} )}}} & (320)\end{matrix}$

A corresponding “mother wavelet” can be defined as $\begin{matrix}{{\psi(x)} = {C_{M,\sigma}\lbrack \quad{{{\mathbb{e}}^{\frac{- x^{2}}{2\sigma^{2}}}\frac{\sin\lbrack {( {M + \frac{1}{2}} )2\quad\pi\quad\frac{x}{L}} \rbrack}{2\quad{\sin( \frac{\pi\quad x}{L} )}}} - {\frac{{\mathbb{e}}^{\frac{- x^{2}}{2\sigma^{2}a^{2}}}}{a}\frac{\sin\lbrack {( {M + \frac{1}{2}} )2\quad\pi\quad\frac{x}{a\quad L}} \rbrack}{2\quad{\sin( \frac{\pi\quad x}{a\quad L} )}}}} \rbrack}} & (321)\end{matrix}$

Because of the constraint on ψ(0), Equation (320), it is verified thatψ(x) is a “small wave”, so its zero frequency transform satisfies$\begin{matrix}{{\hat{\psi}(0)} = {{\int_{- \infty}^{\infty}\quad{{\mathbb{d}x}\quad{\psi(x)}}} = 0}} & (322)\end{matrix}$

The computational usefulness of the DDGWD was shown by several exampleapplications to the solution of differential equations [5,11-14]. Amultiresolution analysis has been developed based on these wavelets[4,6,7].

In this disclosure we enquire as to whether these regularizedinterpolation DAFs can also be obtained in a systematic manner from thesame variational principle [3] used for the noninterpolating DAFs;especially the HDAF [2,3]. We shall see that the DGWD can indeed beobtained directly from our variational principle, and the derivationbears a similarity to that used for the Hermite DAFs. In the nextSection, we give the detailed derivation of the DGWD from thevariational principle.

VARIATIONAL PRINCIPLE APPLIED TO THE DIRICHLET DAF

A general construction of the DAF approximation to a function proceedsby first developing a suitable approximation to the function at everypoint x in its domain. This is typically accomplished by making a basisset expansion of the form $\begin{matrix}{{f( x^{\prime} \middle| x )} = {\sum\limits_{j}{\quad{B_{j}( x^{\prime} \middle| x )}{b_{j}(x)}}}} & (323)\end{matrix}$

Here f(x′{x) is an approximation to the function f(x′) about the pointx, i.e., parameterized by x. The quantity B_(j)(x′{x) is the j th basisfunction for the point x and b_(j)(x) is the corresponding coefficientof this basis function for the expansion centered on the point x. Thecoefficients b_(j)(x) remain to be determined as functionals of theknown values of f(x). A succinct expression for the DAF approximationcan then be given byf_(DAF)(x)=f(x|x)  (324)(although, as previously mentioned, other, more general, definitions,e.g. as parameterized delta sequences, are also possible [4]). Tocomplete the definition one must specify how the x-dependentcoefficients are to be obtained.

There are various ways that the set of coefficients {b_(j)(x) can bedetermined. Perhaps the most straightforward is by the technique of“moving least squares”. In this approach one defines a variationalfunction λ(x) for the point x of the form $\begin{matrix}{{\lambda(x)} = {\sum\limits_{l}\quad{{\omega( {x_{l} - x} )}{{{f( x_{l} \middle| x )} - {f( x_{l} )}}}^{2}}}} & (325)\end{matrix}$where the summation over l is over all points in the domain of x wherethe value of the function is known. (We replace the sum by an integralover all continuous regions of the domain where the function is known.)The quantity ω(x₁—x) is a weight function of arbitrary form, restrictedonly in that it is non-negative. For concreteness we will take ω to beof the Gaussian formω(x)=e^(—x 2)/_(2σ2)   (326)where σ is a parameter with units of length. It should be pointed outthat, in general, the form of the weight can also vary as a function ofx as can the basis functions themselves in both type and number. (Forexample, we could make (a vary with x, and/or choose B_(j)(x′|x) fromdifferent complete sets for each distinct value of x.) We then write$\begin{matrix}{{f( x_{k} \middle| x )} =  {\sum\limits_{j}{B_{j}( x_{k} }} \middle| {x \quad ){b_{j}(x)}} } & (327)\end{matrix}$and determine the optimal values of these coefficients at a particularvalue of x by minimizing the “cost” function λ(x). In general theexpansion coefficients can be complex. In anticipation of thiseventuality, we minimize the cost function with respect to both thecoefficients and their complex conjugates to obtain 2N equations tosolve for the real and imaginary parts ofthe N coefficients. This leadsto the set of linear equations $\begin{matrix}{{\sum\limits_{l = {- \infty}}^{\infty}{{\omega( {x_{l} - x} )}{B_{j}^{*}( x_{l} \middle| x )}{f( x_{l} )}}} = {\sum\limits_{j^{\prime}}{\{ {\sum\limits_{l = {- \infty}}^{\infty}{{\omega( {x_{l} - x} )}{B_{j}^{*}( x_{l} \middle| x )}{B_{j^{\prime}}( x_{l} \middle| x )}}} \} b_{j^{\prime}}}}} & (328)\end{matrix}$which can be written compactly as $\begin{matrix}{A_{j} = {\sum\limits_{j}{C_{{jj}^{\prime}}b_{j^{\prime}}}}} & (329)\end{matrix}$by defining $\begin{matrix}{A_{j} =  {\sum\limits_{l = {- \infty}}^{\infty}{{\omega( {x_{l} - x} )}B_{j}^{*}( x_{l} }} \middle| {x \quad ){f( x_{l} )}} } & (330)\end{matrix}$and $\begin{matrix}{C_{{jj}^{\prime}} =  {\sum\limits_{l = {- \infty}}^{\infty}{{\omega( {x_{l} - x} )}B_{j}^{*}( x_{l} }} \middle| {x \quad )B_{j^{\prime}}( x_{l} } \middle| {x \quad )} } & (331)\end{matrix}$

We immediately have that $\begin{matrix}{b_{j} = {\sum\limits_{j^{\prime}}{( {\underset{\underset{—}{\_}}{C}}^{- 1} )_{{jj}^{\prime}}A_{j^{\prime}}}}} & (332)\end{matrix}$

It is important to recall that all of the quantities in this equationare implicit functions of x.

To proceed we confine our discussion to functions on the real line andrepresent f(x′|x), our local approximation to the function centered onthe point x, as a Fourier series.

The basis functions then are $\begin{matrix}{{B_{j}( x^{\prime} \middle| x )} = {\frac{1}{\sqrt{N}}{\mathbb{e}}^{{- 2}\quad\pi\quad{i{({x^{\prime} - x})}}{j/N}\quad\Delta}}} & (333)\end{matrix}$where j assumes the N values —(N−1)/2≦j≦(N−1)/2. (Note that j takes oninteger values for odd N and half-integer values for even N.) Here Δ isthe grid spacing, which is assumed to be uniform. As a function ofx′,f(x′|x) is obviously periodic with a period domain of NΔ. FromEquation (331) it is seen that C is a kind of overlap matrix for thebasis functions centered at x under the weight function ω(x′—x), and dueto the periodicity of the basis functions, we can express it solely as afunction of η=mod_(Δ)(x′—x). As we now show, it is possible to invertthe matrix C(η) in closed form. However, it is an approximation to theinverse that, when valid, gives rise to the DAF representation of thefunction which is of interest to us here.

It is useful to write the sum in Equation (330) in the form$\begin{matrix}{{\sum\limits_{l = {- \infty}}^{\infty}(\quad)_{l}} = {\sum\limits_{p = {- \infty}}^{\infty}{\sum\limits_{q}(\quad)_{p,q}}}} & (334)\end{matrix}$where 1=Np+q. Here we have divided the grid into domains, each with Npoints. The p-sum is over all domains and the q-sum is over all pointswithin a given domain. We take the point of origin (i.e., p=0, q=0) tobe the grid point closest to x. Thenx_(l)−x=(Np+q)Δ+η,  (335)where −Δ/2≦η≦Δ/2. That is x+η is the grid point closest to x. This leadsto $\quad\begin{matrix}{C_{{jj}^{\prime}} = {{\frac{1}{N}{\sum\limits_{p = {- \infty}}^{\infty}{\sum\limits_{q = 0}^{N - 1}{{\omega( {x_{l} - x} )}{\mathbb{e}}^{2\quad\pi\quad{i{({q + {\eta/\Delta}})}}{{({j - j^{\prime}})}/N}}}}}} = {\sum\limits_{q = 0}^{N - 1}{\lambda_{q + {\eta/\Delta}}\psi_{j}^{({q + {\eta/\Delta}})}\psi_{j^{\prime}}^{{({q + {\eta/\Delta}})}^{*}}}}}} & (336)\end{matrix}$where $\begin{matrix}{\lambda_{q + {\eta/\Delta}} = {\sum\limits_{p = {- \infty}}^{\infty}{\omega( {\lbrack {{N\quad p} + q + {\eta/\Delta}} \rbrack\Delta} )}}} & (337)\end{matrix}$and $\begin{matrix}{\psi_{j}^{({q + {\eta/\Delta}})} = {\sqrt{\frac{1}{N}}{\mathbb{e}}^{2\quad\pi\quad{i{({q + {\eta/\Delta}})}}{j/N}}}} & (338)\end{matrix}$

The quantity ψj(q+ηΔ) can be taken as the jth component of anorthonormal basis set of N-vectors indexed on q. That is $\begin{matrix}{{{\sum\limits_{j = {{- {({N - 1})}}/2}}^{{({N - 1})}/2}{\psi_{j}^{{({q + {\eta/\Delta}})}^{*}}\psi_{j}^{({\overset{\_}{q} + {\eta/\Delta}})}}} = \delta_{q,\overset{\_}{q}}},} & (339)\end{matrix}$which is a standard result from Fourier theory. From this point of view,Equation (327) simply gives an expression for the Cjj matrix element ofC(ζ)in its spectral representation. (Here we have indicated explicitlythat the matrix is a function of ζ.) From this it follows immediatelythat $\begin{matrix}{( {\underset{\underset{—}{\_}}{C}(\eta)}^{- 1} )_{{jj}^{\prime}} = {\sum\limits_{q}{\frac{1}{\lambda_{q + {\eta/\Delta}}}\psi_{j}^{({q + {\eta/\Delta}})}\psi_{j^{\prime}}^{{({q + {\eta/\Delta}})}^{*}}}}} & (340)\end{matrix}$and, further, from Equation(332) that $\quad\begin{matrix}{{b_{j}(\eta)} = {{\sum\limits_{j^{\prime}}{( {\underset{\underset{—}{\_}}{C}(\eta)}^{- 1} )_{{jj}^{\prime}}A_{j^{\prime}}}} = {\sum\limits_{j^{\prime}}{\sum\limits_{q}{\frac{1}{\lambda_{q + {\eta/\Delta}}}\psi_{j}^{({q + {\eta/\Delta}})}\psi_{j^{\prime}}^{{({q + {\eta/\Delta}})}^{*}}{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}{{\omega( {x_{l^{\prime}} - x} )}\psi_{j^{\prime}}^{({q^{\prime} + {\eta/\Delta}})}{f( x_{l^{\prime}} )}}}}}}}} & (341)\end{matrix}$

The sum over j′ here produces the Kronecker δ_(qq′), where$\begin{matrix}{{q^{\prime} + {\eta/\Delta}} = {{\underset{N}{mod}( \frac{x_{l^{\prime}} - x}{\Delta} )}.}} & (342)\end{matrix}$

Summing over q then leads to $\begin{matrix}{{{b_{j}(\eta)} = {\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{\frac{1}{\lambda_{({q^{\prime} + {\eta/\Delta}})}}\quad{\omega( {x_{l^{\prime}} - x} )}\psi_{j}^{({q^{\prime} + {\eta/\Delta}})}{f( x_{l^{\prime}} )}}}},} & (343)\end{matrix}$which is the desired variational expression for the expansioncoefficients. Finally, from Equations(324), (327) and (333) we have that$\begin{matrix}{{{f_{DAF}(x)} = {{\frac{1}{\sqrt{N}}{\sum\limits_{j = {{- {({N - 1})}}/2}}^{{({N - 1})}/2}\quad{b_{j}(\eta)}}} = {\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{\frac{1}{\lambda_{q^{\prime} + {\eta/\Delta}}}\quad{{\omega( {x_{l^{\prime}} - x} )}\lbrack {\frac{1}{\sqrt{N}}{\sum\limits_{j = {{- {({N - 1})}}/2}}^{{({N - 1})}/2}\quad\psi_{j}^{({q^{\prime} + {\eta/\Delta}})}}} \rbrack}{f( x_{l^{\prime}} )}\quad{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{{\frac{( {- 1} )^{q^{\prime}}}{\lambda_{q^{\prime} + {\eta/\Delta}}}\lbrack \frac{\sin( {\pi\quad{\eta/\Delta}} )}{N\quad{\sin( \frac{\pi( {q^{\prime} + {\eta/\Delta}} )}{N} )}} \rbrack}{\omega( {x_{l^{\prime}} - x} )}{f( x_{l^{\prime}} )}}}}}}},} & (344)\end{matrix}$which is the formal-DAF expression without approximation.

If f(x) is periodic with period NΔ, then f(x_(1′)) depends on only q′(i.e., not on p′) and the final result of Eq.(30) reduces to$\begin{matrix}{= {\sum\limits_{q^{\prime} = 0}^{N - 1}{\quad{( {- 1} )^{q^{\prime}}\lbrack \frac{\sin( \frac{\pi\quad\eta}{\Delta} )}{N\quad{\sin( \frac{\pi( {q^{\prime} + {\eta/\Delta}} )}{N} )}} \rbrack}{f( {{q^{\prime}\Delta} + \eta + x} )}}}} & (345)\end{matrix}$where we have made use of Equations (335) and (337). (Recall that x+η isthe grid point closest to x.) This is just the standard Fourierapproximation to a periodic function known on N equally spaced gridpoints. It is interpolative (i.e.,f_(DAF)(x₁)=f(x₁)) where x₁ is anygrid point and hence for which η=0. This is, of course, the anticipatedresult for a least-squares fit of a periodic function using a Fourierbasis.

If f(x) is not periodic then f_(DAF)(x) is nowhere exact (unlessaccidentally so), and, in particular, the DAF approximation is notinterpolative. The quantity f(q′Δ+η+x) in Equation (345) is replaced by$\begin{matrix}{{\overset{\_}{f}}_{q^{\prime}} = {\sum\limits_{p^{\prime} = {- \infty}}^{\infty}\quad{\frac{\omega( {x_{l^{\prime}} - x} )}{\lambda_{q^{\prime} + {\eta/\Delta}}}\quad{f( x_{l^{\prime}} )}}}} & (346)\end{matrix}$which is a weighted average across the infinite grid of functionalvalues on grid points separated by multiples of NΔ.

It is clear that the DAF approximation of Equation (344) (beingbasically a Fourier sum) suffers from the principal drawback of theFourier representation, namely that the approximation is not tightlybanded. That is (off the grid) all of the N values of {overscore(f)}_(q) contribute more or less equivalently to the approximation. Saidanother way, each grid point contributes through the normalizedprobability ω(x_(l)−x)/λ_(q+η/Δ), which falls off much more slowly thanω(x_(l)−x) itself as q is varied. To introduce a more tightly banded DAFrepresentation of the function, we now assume that C(η) can beeffectively replaced by a matrix that is independent of η. In so doingwe, of course, ignore variations in C(η) over the distance of the gridspacing. There are various ways that this can be done. In previousstudies, where we employed a polynomial basis set rather than thecircular functions of Equation( 333), it proved convenient to replaceC(η) by its average. This allowed us to use the properties of orthogonalpolynomials to construct the corresponding approximation to C ⁻¹. Wereferred to the resulting representation of the function aswell-tempered because it has the property that for functions where theapproximation is applicable (so-called DAF-class functions) the fit isof comparable accuracy both on and off the grid. In contrast, in thepresent case it is convenient to make an η-independent approximation toC(η) for which the grid points are special.

The idea is that as N becomes large and the grid spacing becomes smallin such a way that N/Delta is held constant, every point becomeseffectively a grid point (assuming continuity of the function to befit). Then, to controllable accuracy we can replace C(η) by C(η=0) toobtain $\begin{matrix}{( {\underset{\underset{—}{\_}}{C}(\eta)}^{- 1} )_{{jj}^{\prime}} \approx {\sum\limits_{q}{\frac{1}{\lambda_{q}}\psi_{j}^{(q)}\psi_{j^{\prime}}^{{(q)}^{*}}}}} & (347) \\{\quad{b_{j} \approx {\sum\limits_{j^{\prime}}{\sum\limits_{q}{\frac{1}{\lambda_{q}}\psi_{j}^{(q)}\psi_{j^{\prime}}^{{(q)}^{*}}{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}{{\omega( {x_{l^{\prime}} - x} )}\psi_{j}^{({q^{\prime} + {\eta/\Delta}})}{f( x_{l^{\prime}} )}}}}}}}} & (348)\end{matrix}$and $\begin{matrix}{{{f_{DAF}(x)} \approx {\frac{1}{\lambda_{0}}{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{{\omega( {x_{l^{\prime}} - x} )}\frac{1}{\sqrt{N}}{\sum\limits_{j^{\prime} = {{- {({N - 1})}}/2}}^{{({N - 1})}/2}\quad{\psi_{j^{\prime}}^{({q^{\prime} + {\eta/\Delta}})}{f( x_{l^{\prime}} )}}}}}}} = {\frac{\Delta}{\lambda_{0}L}{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{{\omega( {x_{l^{\prime}} - x} )}{\sum\limits_{j^{\prime} = {{- {({N - 1})}}/2}}^{{({N - 1})}/2}{{\mathbb{e}}^{{- 2}\pi\quad{i{({x_{l^{\prime}} - x})}}{j^{\prime}/L}}{f( x_{l^{\prime}} )}}}}}}} & (349)\end{matrix}$where L=NΔ. The applicability of this approximation depends of course onthe appropriate choice of the DAF parameters, which has been discussedelsewhere. The sum can be written in terms of the Mth order Dirichletkernel, D_(M)(y), defined by $\begin{matrix}{{D_{M}(y)} = {{\frac{1}{\pi}\lbrack {\frac{1}{2} + {\sum\limits_{k = 1}^{M}\quad{\cos({ky})}}} \rbrack} = \frac{\sin\lbrack {( {M + \frac{1}{2}} )y} \rbrack}{2\pi\quad{\sin( {y/2} )}}}} & (350)\end{matrix}$which leads to the expression $\begin{matrix}{{f_{DAF}(x)} = {\frac{2\pi\quad\Delta}{\lambda_{0}L}{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{{\omega( {x_{l^{\prime}} - x} )}{D_{\frac{N - 1}{2}}( \frac{2{\pi( {x_{l^{\prime}} - x} )}}{L} )}{f( x_{l^{\prime}} )}}}}} & (351)\end{matrix}$

This result is parameterized by three quantities L, N (which are relatedby the grid spacing Δ) and σ (see the form of ω of Equation (326).

Since our approximation C(η)≈C(η=0) is exact on the grid, thisapproximation is interpolative for functions that are periodic on adomain of length L. If we take the limit N→∞ and L→∞ in such a way thatΔ=L/N is fixed, then the approximation assumes the sinc-DAF form$\begin{matrix}{{f_{DAF}(x)} = {\frac{\Delta}{2\pi}{\sum\limits_{l^{\prime} = {- \infty}}^{\infty}\quad{{\omega( {x_{l^{\prime}} - x} )}\frac{\sin( {2{{\pi( {x_{l^{\prime}} - x} )}/\Delta}} )}{( {x_{l^{\prime}} - x} )}{f( x_{l^{\prime}} )}}}}} & (352)\end{matrix}$where we have used the fact that λ0 =1 in this limit. This result isinterpolative on all grid points.

CONCLUSION

We have shown that the variational principle used earlier for generatingnoninterpolating DAFs (which could be used to generate associatedwavelets) can also be used to derive interpolating DAFs, with a Gaussianweight, that were first obtained by multiplying various interpolationshells with a Gaussian, which regularized the function (making itinfinitely differentiable) and ensured that it decays rapidly both inphysical and Fourier space. We therefore conclude that the interpolatingand noninterpolating DAFs are very closely related, corresponding todifferent ways of solving the moving least squares variational algebraicequations. This result complements the earlier procedure used toconstruct the interpolating DAFs and provides another framework in whichto develop robust approximation and estimation algorithms. Both theinterpolating and noninterpolating DAFs, of course, have been shownpreviously to be computationally robust. [1-2,6-7,11-16]

REFERENCES

-   -   [1] G. W. Wei, D. J. Kouri and D. K. Hoffman, Computer Phys.        Commun., 112, 1 (1998).    -   [2] D. K. Hoffman, N. Nayar, O. A. Sharafeddin and D. J.        Kouri, J. Phys. Chem., 95, 8299 (1991); D. J. Kouri, W. Zhu, X.        Ma, B. M. Pettitt, and D. K. Hoffman, ibid., 96, 1179 (1992).    -   [3] D. K. Hoffman, T. L. Marchioro, M. Arnold, Y. Huang, W. Zhu,        and D. J. Kouri, J. Math. Chem. 20, 117 (1996).    -   [4] Y. Meyer, Wavelets and Operators, Cambridge Stud. Adv.        Math.,vol. 37 (Cambridge Univ. Press, Cambridge, UK, 1992).    -   [5] I. Daubechies, Ten Lectures on Wavelets, CBMS-NSF Series in        Applied Mathematics (SIAM, Philadelphia, 1992).    -   [6] C. K. Chui, An Introduction to Wavelets (Academic Press, San        Diego, Calif., 1992).    -   [7] G. W. Wei, D. J. Kouri, and D. K. Hoffman, to be published.    -   [8] G. W. Wei, D. S. Zhang, D. J. Kouri and D. K. Hoffman, Phys.        Rev. Lett. 79, 775 (1997).    -   [9] Z. Shi, D. J. Kouri, G. W. Wei, and D. K. Hoffman, Computer        Phys. Commun., in press.    -   [10] Z. Shi, G. W. Wei, D. J. Kouri, and D. K. Hoffman, IEEE        Symp. on Time-frequency and Time-scale Analysis}, N. 144, pp.        469-472, Pittsburgh, Pa., Oct. 6-9, 1998.    -   [11] G. W. Wei, D. S. Zhang, D. J. Kouri, and D. K. Hoffman, J.        Chem. Phys. 107, 3239 (1997).    -   [12] D. S. Zhang, G. W. Wei, D. J. Kouri, and D. K. Hoffman,        Phys. Rev. E 56, 1197 (1997).    -   [13] G. W. Wei, S. C. Althorpe, D. J. Kouri, and D. K.        Hoffman, J. Chem. Phys. 108, 7065 (1998).    -   [14] G. W. Wei, S. C. Althorpe, D. S. Zhang, D. J. Kouri,        and D. K. Hoffman, Phys. Rev. A 57, 3309 (1998).    -   [15] D. K. Hoffman and D. J. Kouri, in Proc. 3^(rd). Internat.        Conf. on Math. and Num., Aspects Wave Prop. ed. G. Cohen (SIAM,        Philadelphia, 1995), pp. 56-63.    -   [16] A. Frishman, D. K. Hoffman, and D. J. Kouri, J. Chem.        Phys., 107, 804 (1997).

WAVELETS, DELTA SEOUENCES AND DAFs Introduction

The rapid development and great success of wavelet theory and technologyin the last decade [1-7] have stimulated intense interest amongmathematicians, engineers, physicists and chemists. New results areregularly reported and applications are found in virtually everydiscipline of science and engineering [8-10]. The basic theory andconstruction procedures are regarded as well understood. However, thereare areas where the existing wavelet methods encounter difficulties. Onesuch area is computational fluid dynamics and more generallycomputational chemistry and physics, and in mechanics. Meyer [4] hasrecently posed the questions “Can wavelets play a part inthe study orthe understanding of the Navier-Stokes equations?” He concluded that “westill do not know the answer to this question”. Since wavelets areintimately and significantly related to spline theory and the theory ofapproximations, it is likely that wavelet theory can lead to entirelynew approaches for scientific and engineering computations, wheretraditional method are either global or local. It is well known thatglobal spectral methods are accurate and efficient for linear partialdifferential equations (PDEs), whereas local methods are simple andconvenient for nonlinear PDEs. It is extremely important to develop anapproach that delivers global method accuracy, while also providinglocal method flexibility and simplicity, for nonlinear PDEs involvingsingularities, or homoclinic orbits, for which obtaining accurate andstable numerical solutions is still a major challenge [11,12]. Wavelettheory has been intensively studied for this purpose [13-23].

The rapid growth and unprecedented success of wavelets have led mostrecent researchers to focus more on the exploration of new applicationsrather than reflecting on the basic concepts. The predominant view ofwavelets has been strongly influenced by the belief that there areanalysis tools for describing a signal efficiently in time and infrequency simultaneously, thus overcoming the classical limitation ofFourier analysis, which is strictly efficient either in the time or thefrequency domain. As pointed out by Flandrin and Goncalvès [24],wavelets and wavelet transforms are basically time-scale tools, ratherthan time-frequency ones. Moreover, as far as time-frequency analysis isconcerned, there are many other approaches which are able to outperformboth Fourier analysis and wavelets in certain cases. There are otherviews, which are somewhat less influential and less developed. Forexample, Holschneider has recognized that a real-valued, non-negativescaling wavelet provides a smoothed version of a function ƒ over thereal line R by means of the convoluation product $\begin{matrix}{{\Phi_{a,b}(f)} = {{\langle {{\phi_{a}(b)},f} \rangle{\int_{- \infty}^{\infty}{{\mathbb{d}t}\quad\frac{1}{a}\phi\quad( \frac{b - t}{a} ){f(t)}}}} \equiv {{\phi_{a}(b)}*f}}} & (353)\end{matrix}$

Then a wavelet transform given by $\begin{matrix}{{W_{a,b}(f)} = {{- {\partial_{a}\quad{\int_{- \infty}^{\infty}{{\mathbb{d}t}\quad\frac{1}{a}\phi\quad( \frac{b - t}{a} )\quad{f(t)}}}}} = {{\int_{- \infty}^{\infty}{{\mathbb{d}t}\quad\psi\quad( \frac{b - t}{a} )\quad{f(t)}}} \equiv {{\psi_{a}(b)}*f}}}} & (354)\end{matrix}$

Where ψ(x)=(x∂+1)φ(x) and ψ_(α)(x/α)/α, provides a mathematicalmicroscopy of ƒ at length scale a. These are important ideas and theydeserve additional study.

In a separate development, Hoffman, Kouri and co-workers [23-29] havepresented a powerful computational method based on distributedapproximating functionals (DAFs), for various numerical applications,including solving linear [30] and nonlinear [31] partial differentialequations (PDEs), signal analysis and the padding of experimental data.On the real line R, the DAFs are multiparameter delta sequences of theDirichlet type, constructed using functions of the Schwartz class. Thesuccess of both wavelets and DAF's in a variety of applications has ledto a search for a connection between them. Indeed, there is a naturalconnection between the [32] and numerous wavelet-DAFs and DAF-waveletshave been discovered [32,33]. Wavelet-DAFs have been used to obtainresults in solving the Navier-Stokes equations [34] with nonperiodicboundary conditions. In fact, underlying both wavelets and DAFs is acommon mathematical structure, the theory of distributions, which wasinitiated by physicists and engineers, and was later presented in arigorous mathematical for by Schwartz [35]. Korevaar [36] and others. Ageneral distribution analysis of wavelets has been given by Meyer [1],Meyer and Coifman [4]. Daubechies [3] and others [37]. Orthogonalwavelet expansions of the delta distribution are discussed by Walter[38]. However, the role of delta sequences in wavelet theory has hardlybeen addressed. In particular, a large class of delta sequences can beidentified as scale wavelets, and they can be used as multi-resolutionwavelet bases, which we call delta-sequence-generated multi-resolutionwavelet bases (or DAF-wavelets). The purpose of the present work is toprovide a unified description of wavelets, delta sequences and DAFs.

REVIEW OF DELTA SEQUENCES

The delta distribution or so-called Dirac delta function began with theHeaviside calculus and was informally used by physicists and engineersbefore Sobolev, Schwartz [35], Korevaar [36] and others put it into arigorous mathematical form. There are several formal mathematicalconstructions which have been used in the literature (35,36). We shalltake the approach which obtained the delta function as a sequence limit(2).

A sequence of functions δ_(α)(x)∈−L₁φ(x), is a delta sequence on thedomain J if for each x∈J and suitable functions $\begin{matrix}{{\lim\limits_{\alphaarrow\alpha_{0}}{\int_{J}{{\delta_{\alpha}(x)}{\phi(x)}{\mathbb{d}x}}}} = {\phi(0)}} & (355)\end{matrix}$where the sequences δ_(α)are generalizations of Cauchy sequences and arecalled fundamental families on J by Korevaar [36].

Two types of delta sequences (the positive type and the Dirichlet type)can be used as the basis of a useful classification scheme. Both typesare discussed in the following two subsections. (Moreover, classifyingdelta sequences according to whether they belong to the Schwartz classor non-Schwartz class is also very useful for various physical andengineering applications.)

Delta Sequences of the Positive Type

Let {δ_(α)} be a sequence of functions on (−∞,∞) which are integrableover every bounded interval. A delta sequence {δ_(α)} is of positivetype if, for any bounded interval containing the origin, the integral inthe α→α₀ limit of the sequence of the functions equals 1. For boundedintervals excluding the origin, the integral of the sequence offunctions equals zero in this limit. All members of the sequence arepositive semi-definite. To illustrate, we consider first the example ofa delta sequence of impulse function.

To approximate idealized physical concepts such as the force density ofa unit force at the origin x=0, or a unit impulse at time x=0, one has$\begin{matrix}{{\delta_{\alpha}(x)} = \{ \begin{matrix}\alpha & {{{for}\quad 0} < x < {1/\alpha}} & {{\alpha = 1},2,\ldots} \\0 & {otherwise} & \quad\end{matrix} } & (356)\end{matrix}$as a delta sequence in the limit α→∞.

A second example is Gauss' delta sequence, given by $\begin{matrix}{{\delta_{\alpha}(x)} =  {\frac{1}{\sqrt{\pi\quad}\alpha}{\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}\quad{for}\quad\alpha}arrow 0 } & (357)\end{matrix}$and it arises naturally as a distribution solution or so-called weaksolution. Gauss' delta sequence has various interesting properties withregard to differentiability, boundedness and Fourier transforms, and isused to generate the “Mexican hat” wavelet. Various other wavelets canalso be generated from it as described in later sections of this paper.

A third example is Lorentz's delta sequence, $\begin{matrix}{{\delta_{\alpha}(x)} =  {\frac{1}{\pi}\frac{\alpha}{x^{2} + \alpha^{2}}{for}\quad\alpha}arrow 0 } & (358)\end{matrix}$which is used in representing the solution of Laplace's equation in theupper half plane. It is also commonly seen in integral equationsinvolving the Green's function of the kinetic energy operator (in themomentum representation).

Other examples included Landau's, Poisson's and Fejer's delta sequences.

A systematic procedure for generating various delta sequences of thepositive type is a follows:ρα(x)={fraction (1/α)}ρ({fraction (x/α)})(α>0)  (359)Delta Sequences of the Dirichlet Type

Definition 4. Let {δα} be a sequence of functions o (−∞, ∞) which - - -every bounded internal. We call {δα} a delta sequence of the Dirichlet

-   -   1. ∫δα→1 as α→α₀ For some finite constant α.    -   2. For every constant        γ > 0, (∫_(−∞)^(γ)  +∫_(γ)^(∞))δ_(α) → 0  as  α → α₀.    -   3. There are positive constants—and—such that        ${{\delta_{\alpha}(x)}} \leq {\frac{C_{1}}{x} + C_{2}}$        for all x and α.

EXAMPLE 1 Dirichlet's Delta Sequence

An important example of a delta sequence of the Dirichlet type isDirichlet's delta sequence $\begin{matrix}{{\delta_{\alpha}(x)} = \{ \begin{matrix}{D_{\alpha}(x)} & {{{for}\quad{x}} < \pi} & {{{{for}\quad\alpha} = 0},1,{2\quad\ldots}} \\0 & {{otherwise},} & \quad\end{matrix} } & (360)\end{matrix}$where D_(α)is the Dirichlet kernel given by Equation (350). Dirichlet'sdelta sequence plays an important role in approximation theory and isthe key element in trigonometric polynomial approximations.

EXAMPLE 2 Modified Dirichlet's Delta Sequence

Sometimes there is a slight advantage in taking the last term in D_(α)with a factor of ½: $\begin{matrix}{{{{D_{\alpha}^{*}(x)} = {{D_{\alpha} - {\frac{1}{2}{\cos( {\alpha\quad x} )}}} = \frac{\sin( {\alpha\quad x} )}{2\quad\pi\quad{\tan( {x/2} )}}}},{\alpha = 0},1,2,\ldots}\quad} & (361)\end{matrix}$This is the so-called modified Dirichlet kernel. The differenceD_(α)−D_(α)* tends uniformly to zero on (−π, π) as α→∞. They areequivalent with respect to convergence. The expression given by$\begin{matrix}{{\delta_{\alpha}(x)} = \{ \begin{matrix}{D_{\alpha}^{*}(x)} & {{{for}\quad{x}} < \pi} & {{{{for}\quad\alpha} = 0},1,{2\quad\ldots}} \\0 & {otherwise} & \quad\end{matrix} } & (362)\end{matrix}$is a delta sequence of Dirichlet type as α→∞.

EXAMPLE 3 Dirichlet's continuous delta sequence

Dirichlet's continuous delta sequence is given by the following Fouriertransform of the characteristic function ω[−α,α]. $\begin{matrix}{{\delta_{\alpha}(x)} = {{\frac{1}{2\quad\pi}{\int_{- \infty}^{\infty}{{\chi\lbrack {{- \alpha},\alpha} \rbrack}{\mathbb{e}}^{{- i}\quad\xi\quad x}{\mathbb{d}\xi}}}} = \frac{\sin( {\alpha\quad x} )}{\pi\quad x}}} & (363)\end{matrix}$

This converges to the delta distribution as α→∞. Equation (363) isrelated to Shannon's sampling theorem in information theory.

EXAMPLE 4 The de la Vallee Poussin Delta Sequence

The de la Vallee Poussin delta sequence is given by the Fourier inverseof the following function $\begin{matrix}{{{\hat{\delta}}_{\alpha}(\xi)} = \{ \begin{matrix}1 & {{\xi } \leq \alpha} \\{2 - {\xi/\alpha}} & {< {\xi } \leq {2\quad\alpha}} \\0 & {otherwise}\end{matrix} } & (364)\end{matrix}$It is easy to show that $\begin{matrix}{{\delta_{\alpha}(x)} = {\frac{1}{\pi\quad\alpha}\quad\frac{{\cos( {\alpha\quad x} )} - {\cos( {2\quad\alpha\quad x} )}}{x^{2}}}} & (365)\end{matrix}$is a delta sequence of the Dirichlet type as α→∞.

EXAMPLE 5 Interpolative Delta Sequence

Let {δ_(n)} be a sequence and let {x_(i)}₀ ^(n) be n+1 zeroes of aJacobi polynomial in (a, b). $\begin{matrix}{{{\Delta_{n}( {x,y} )} = {\frac{\prod\limits_{i = 0}^{n}( {x - x_{i}} )}{( {x - y} ){\prod\limits_{i = 0}^{n}( {y - x_{i}} )}}{\sum\limits_{i = 0}^{n}{{\delta_{n}( {y - x_{i}} )}\quad x}}}},{y \in ( {a,b} )}} & (366)\end{matrix}$is a delta sequence as n→∞. This follows from the fact that∫Δ_(n)(x,y)f(y)dy approximations to the Lagrange interpolating formulaewhich converges as n→∞ and sup

EXAMPLE 6 Delta sequences Constructed by Orthogonal Basis Expansions

Let {ψ_(n)} be a complete orthonormal L²(a,b) basis. Then$\begin{matrix}{{{\delta_{n}( {x,y} )} = {\sum\limits_{i = 0}^{n}{{\phi_{i}(x)}{\phi_{i}(y)}\quad x}}},{y \in ( {a,b} )}} & (367)\end{matrix}$are delta sequences. In case of trigonometric functions, we again obtainthe Dirichlet kernel delta sequences given in the Examples 1 and 3. AHermite function expansion is given by $\begin{matrix}{{{\delta_{n}(x)} = {{\exp( {- \frac{x^{2}}{2}} )}{\sum\limits_{k = 0}^{n}{( {- \frac{1}{4}} )^{k}\frac{1}{\sqrt{2\quad\pi}\quad{k!}}{H_{2\quad k}( \frac{x}{\sqrt{2}} )}}}}},{\forall{\in R}}} & (368)\end{matrix}$where H_(2k)(π/√2) is the usual Hermite polynomial. This delta sequencewas studied by Schwartz [44], Korevaar [36] and was independentlyrediscovered by Hoffman, Kouri and coworkers [25] in a more generalform. Various other cases can be found in Walter and Blum's reference[39] and these also have been studied in very general forms by Hoffman,Kouri and coworkers [28].

WAVELETS AND DELTA SEQUENCES

Wavelets have been widely used as an analysis tool for variousapplications. The essential reason for this is because both orthogonaland nonorthogonal wavelets can provide a decomposition of a function ata variety of different scales. In other words, wavelets form special L²(R) bases or frames for representing a function at various levels ofdetail, leading to so-called mathematical microscopies. This turns outto be very efficient for approximating and analyzing functions in manyapplications. Orthogonal wavelets and multiresolution analysis have beensuccessfully used in a variety of telecommunication and engineeringfields [10]. They play a special role in those applications whereorthogonality is strongly required. In many other applications,nonorthogonal wavelets, or frames, are also very useful. Generating newtypes of wavelets has been of great importance in wavelet theory. Mostof the delta sequences described in Section II can be regarded asscaling or father wavelets. These father wavelets can also besystematically transformed into mother wavelets. The orthogonal waveletsare briefly reviewed in the first subsection. The connection betweendelta sequences and wavelets is made in the second subsection. Theconstruction of mother wavelets from various delta sequences isdiscussed in the last two subsections.

Orthogonal Wavelets

The formal theory of orthogonal wavelets on L²(R) has been presented inmany books [1-3]. An orthogonal wavelets system is usual generated by asingle function, either a father wavelet Φ or a mother wavelet ψ, by astandard translation and dilation technique $\begin{matrix}{{{\phi_{mn}(x)} = {2^{{- m}/2}{\phi( {\frac{x}{2^{m}} - n} )}}},\quad m,{{n \in Z};}} & (369) \\{{{\psi_{mn}(x)} = {2^{{- m}/2}{\psi( {\frac{x}{2^{m}} - n} )}}},\quad m,{{n \in Z};}} & (370)\end{matrix}$where the symbol Z denotes the set of all integers. This can beformulated rigorously in terms of a multiresolution analysis, i.e., anested sequence of subspaces {V_(m)}, m∈Z such that

-   -   1. {Φ(x−n))} is an orthogonal basis of V₀;    -   2. . . . ⊂V₁⊂V₀⊂V⁻¹⊂ . . . ⊂L² (R);    -   3. f(x)∈V_(m)←→ƒ(2x)∈V_(m-1);    -   4. ∩_(m)V_(m)={0} and {overscore (∪_(m)V_(m))}=L²(R).        Since Φ∈V₀⊂V⁻¹, it can be expressed as superposition of        {Φ_(1,n)} which constitute basis for V⁻¹ $\begin{matrix}        {{{\phi(x)} = {\sum\limits_{\pi}{a_{n}\phi_{1,n}}}},} & (371)        \end{matrix}$        where {α_(n)} is a set of finite coefficients.

For an orthogonal system, the subspace V⁻¹ can be further decomposedinto its orthogonal projection in V₀ and a complement W₀V ⁻¹ =V ₀ ⊕W ₀,  (372)where W₀ is a subspace spanned by orthogonal mother wavelets {ψ}. Ingeneral, ψ_(min), n∈Z is an orthogonal basis of W_(−m), and$\begin{matrix}{\overset{\_}{\underset{m \in Z}{\oplus}W_{m}} = {{L^{2}(R)}.}} & (373)\end{matrix}$It follows that ψ_(min), (m, n ∈ Z) is an orthogonal basis of L² (R).Similarly to Equation (371), the mother wavelet can also be expressed asa superposition of {Φ_(1,n)}. $\begin{matrix}{{{\psi(x)} = {\sum\limits_{n}{b_{n}\phi_{1,n}}}},} & (374)\end{matrix}$where b_(n)=(−1)^(n)α_(1-n).

Perhaps the simplest example is Haar's wavelet system [2] which is givenby Φ(x)=x_([0,1])(x), the characteristic function of the interval [0,1].It obviously has orthogonal translations. The dilation of Φ (x) resultsin characteristic functions for smaller (or larger) intervals and eachof them spans a subspace V_(m) by translations.

It is not obvious that a multiresolution analysis exists for Φ otherthan the Haar system. The construction of the first few orthogonalwavelet bases was more or less an art rather than a systematicprocedure; it required ingenuity, special tricks and subtlecomputations. One procedure, due to Meyer [1], is to begin with a splinefunction θ(x)=(1−|x−1|)x_([0,2])which, by translations, generates anonorthogonal Riesz basis (a frame of the lease possible redundance.Using both the orthonormality requirement, $\begin{matrix}{\delta_{0,n} = \langle {\phi_{0,0}\phi_{0,n}} \rangle} & (375)\end{matrix}$and the periodicity, φ can be resolved as $\begin{matrix}{{\hat{\psi}(\xi)} = {\frac{\sin^{2}( {\xi/2} )}{( {\xi/2} )^{2}}( {1 - {\frac{2}{3}{\sin^{2}( {\xi/2} )}}} )^{- \frac{1}{2}}}} & (376)\end{matrix}$

Daubechies presents another approach for constructing orthogonalwavelets. In the Fourier representation, the dilation equation can bewritten as{circumflex over (ψ)}(ξ)=m₀(ξ/2){circumflex over (ω)}(ξ/2)  (377)where m₀ is a 2π-periodic function. The orthonormality condition thenrequires|m ₀(ξ/2)|² +|m ₀(ξ/2+π)|²=1  (378)

It turns out that if the set of expansion coefficients α_(n) of Equation(371) are chosen asa ₀=ν(ν−1)/(ν+1)√2, a ₁=−(ν−1)/(ν+1)√2, a ₂=(ν−1)/(ν+1)√2, a₃=ν(ν+1)/(v+1)√2(ν∈R),then Equation (378) will be satisfied and consequently ∅ can be foundrecursively.Delta Sequences as Father Wavelets

Let {δ_(n)} where α→α₀ be a sequence of C^(m) functions on (−∞, ∞) whichare integrable over every bounded interval and

-   -   1. {circumflex over (δ)}_(α()0)=1 for each α;    -   2. lim_(α→α) ₀ δ_(α)(ε)→1 for all δ;    -   3. for every constant γ<0, (∫^(−γ) _(−∞)+∫^(∞) _(γ))δ_(α)→0 as        α→α₀;    -   4. and ∥xδ_(α)(x)∥_(∞)<∞ for all x and α.

Then the {δ_(n)} are delta sequences and each function can be admittedas a father wavelet. We call this class of father wavelets“delta-sequence-generated father wavelets φ_(a)”. The corresponding“delta-sequence-generated mother wavelets ψ_(α)” have the Fouriertransform property{circumflex over (ψ)}_(α)(0)=∫ω_(α)(x)dx  (379)

It is natural to view delta sequences as father wavelets. In particular,if the delta sequence has the structure that δ_(α)=(1/α)ρ(ξ/α) and∫ρ(x)dx=1 as is the case for many examples given in Section II), then67n is a sequence of father wavelets at different scales. In contrast tothe delta distribution which has only a point support, a function in adelta sequence can have an arbitrary support, depending on the scale. Inthe limit α→α₀, the delta sequence converges to the delta distributionand the support shrrinks down to a point. The resulting deltadistribution actually helps to furnish the wavelet multiresolutionanalysis [37] $\begin{matrix}{{{\{ \delta \} \oplus {\underset{m \in \quad Z}{\oplus}W_{m}}} = {L^{2}(R)}},} & (380)\end{matrix}$where {δ} is the space spanned by the delta distribution. This is incontrast to Equation (373). Clearly, if a delta sequence is anorthogonal system, such as Dirichlet's continuous delta sequence, for afixed α≢α₀, we have $\begin{matrix}{{{\{ \delta_{\alpha} \} \oplus {\underset{- \infty}{\overset{m = 0}{\oplus}}W_{m}}} = {{\{ \delta \} \oplus {\underset{m \in Z}{\oplus}W_{m}}} = {L^{2}(R)}}},} & (381)\end{matrix}$where {δ_(n)} spans the wavelet subspace V₀. Hence the orthogonal deltasequence spans the wavelet subspace {δ}⊕⊕₁ ^(∞)W_(m) for an appropriatechoice of α.

Delta sequence generated mother wavelets can be constructed in manydifferent ways. We discuss two approaches in the next two subsections.

Wavelets Generated by Differentiation Pairs

For a given C^(m) father wavelet φ, we define a family of waveletgenerators $\begin{matrix}{{G^{(n)} = {{x\frac{\partial^{n}}{\partial x^{n}}} + {n\frac{\partial^{n - 1}}{\partial x^{n - 1}}}}},\quad{n = 0},1,2,\ldots\quad,m} & (382)\end{matrix}$for generating a family of m+1 mother waveletsΨ_(α,n)(x)=G ^((n))x_(α)(x) for φ∉C^(m) and n=0,1,2, . . . , m  (383)It is noted that this approach is not restricted to thedelta-sequence-generated wavelet and is actually a very general andefficient way for creating wavelets from a given C^(m) father wavelet.The transform prescribed by Holschneider [6], Equation (354), is aspecial case of our family of wavelet generators.

Our wavelet generators are closely related top the transformation Liegroup of translations and dilations. This is because the Fourier imagesof distributions $\begin{matrix}{{G^{n} = {{x\frac{\partial^{n}}{\partial x^{n}}} + {n\frac{\partial^{n - 1}}{\partial x^{n - 1}}}}}\quad,\quad{n = 0},1,2,\ldots} & (384)\end{matrix}$form an infinite C^(m) dimensional wavelet Lie algebra with elements{X_(n)=ξ^(n−1)∂_(ξ)|n=0,1,2, . . . } (here we follow the convention thatstatements concerning the structure of a Lie algebra are made only onthe basis of the real Lie algebra). The whole Lie Algebra structure of{X_(n)} is simply given by[X_(n),X_(m)]=(m−n)X_(m+n−2) n,m=0,1,2, . . .   (385)X₁ generates a one-parameter non-compact abelian group which isobviously the translation group in momentum space. X₂ generates adilation group. There are two nontrivial invariant subalgebras[X ₁ ,X ₂ ]=X ₁; and  (386)[X ₁ ,X ₂ ]=X ₁ ,[X ₁ ,X ₃]=2X ₂ ,[X ₂ ,X ₃ ]=X ₃  (387)X₁, X₂ are respectively the infinitesimal generators of atwo-dimensional translation and dilation group. The third element X₃ isa quadratic dilation (superdilation), which allows us to generateanother invariant subalgebra, Equation (387). This result indicates thatour method of systematically generating wavelets is very general and hasa mathematically rigorous foundation. More details will be presentedelsewhere [45].

EXAMPLE 1 Mexican Hat Wavelet and Generalizations

If we take Gauss's delta sequence as a father wavelet${{\phi_{\alpha}(x)} = {\frac{1}{\sqrt{{2\pi}\quad}\alpha}{\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}}},$then Equation (383), for n=1, is $\begin{matrix}{{{\psi_{\alpha,1}(x)} = {\frac{1}{\sqrt{2\pi}\alpha}( {1 - \frac{x^{2}}{\alpha^{2}}} )\quad{\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}}},} & (388)\end{matrix}$which is the well-known Mexican hat wavelet [3]. Taking n=3 yields$\begin{matrix}{{\psi_{\alpha,3}(x)} = {\frac{- 2}{\sqrt{2\pi}\alpha}( {\frac{x^{4}}{\alpha^{4}} - \frac{6x^{2}}{\alpha^{2}} + 3} ){\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}}} & (389)\end{matrix}$which is an interesting “Mexican superhat wavelet”. This wavelet isexpected to perform better than the Mexican hat for some application.Since elements of Gauss' delta sequence are C^(∞)functions, there areinfinitely many Gauss-delta-sequence-generated wavelets given by$\begin{matrix}{{\begin{matrix}{{\psi_{\alpha,n}(x)} = {{G^{n}\frac{1}{\sqrt{2\pi}\alpha}{\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}} = {\frac{1}{\sqrt{2\pi}\alpha}\frac{( {- 1} )^{n}}{2}{H_{n + 1}( \frac{x}{\sqrt{2}\alpha} )}{\mathbb{e}}^{- \frac{x^{2}}{2\alpha^{2}}}}}} \\\quad\end{matrix}\quad{n = 0}},1,2,{\ldots\quad.}} & (390)\end{matrix}$It is seen that the celebrated Mexican hat wavelet [3] is just a specialcase of Equation (390).

It is interesting to note that all higher order Hermite functions (n≠0)are mother wavelets, while the lowest order Hermite function is a fatherwavelet. This can be naturally seen from the orthonormality condition$\begin{matrix}{{{\int_{- \infty}^{+ \infty}{\frac{1}{\alpha\sqrt{{\pi 2}^{n + m + 1}{n!}{m!}}}\quad{H_{n}( \frac{x}{\sqrt{2}\alpha} )}\quad{H_{m}( \frac{x}{\sqrt{2}\alpha} )}{\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}{\mathbb{d}x}}} = \delta_{nm}},\quad n,\quad{m = 0},1,2,{\ldots\quad.}} & (391)\end{matrix}$Here, if the second polynomial if fixed as m=0 to a given constant, thenthe only case to H_(n) to give a nonzero integration is n=0 whichdetermines a father wavelet. All other H_(n)(n≠0) give rise to motherwavelets. We point out that this result is not limited to Herrnitepolynomial. It is generally true for all polynomial systems which formorthogonal L²(a, b) bases with respect to an appropriate weight. Moredetails about the general connection between wavelets and conventionalHilbert space bases will be given elsewhere [46].

EXAMPLE 2 Shannon's Wavelet Family

Dirichlet's continuous delta sequence is related to the well-knownShannon's father wavelet${\phi_{\alpha}(x)} = {\frac{1}{\pi}\quad{\frac{\sin( {\alpha\quad x} )}{x}.}}$The latter is known for generating an orthogonal basis for a reproducingkernel Hilbert space. A family of (mother) wavelets can be generated byusing our wavelet generations, Equation (382) $\begin{matrix}\begin{matrix}{{\phi_{\alpha,n}(x)} = {( {{x\frac{\partial^{n}}{\partial x^{n}}} + {n\frac{\partial^{n - 1}}{\partial x^{n - 1}}}} )\frac{1}{\pi}\quad\frac{\sin( {\alpha\quad x} )}{x}}} \\{= {{\frac{1}{\pi}\quad{\sin( {\alpha\quad x} )}\quad{for}\quad n} = 0}} \\{= {{\frac{- \alpha}{\pi}\quad{\cos( {\alpha\quad x} )}\quad{for}\quad n} = 1}} \\{= {{\frac{\alpha^{2}}{\pi}\quad{\sin( {\alpha\quad x} )}\quad{for}\quad n} = 2}} \\{= {{\frac{( {- 1} )^{q}\alpha^{2q}}{\pi}\quad{\sin( {\alpha\quad x} )}\quad{for}\quad n} = {2\quad q}}} \\{= {{\frac{( {- 1} )^{q}\alpha^{{2q} + 1}}{\pi}\quad{\cos( {\alpha\quad x} )}\quad{for}{\quad\quad}n} = {{2q} + 1}}}\end{matrix} & (392)\end{matrix}$

These results are in contract to Shannon's wavelet, {fraction(1/πx)}[sin(2πx)−sin(πx)]. Obviously, all of these wavelets can be usedto generate orthogonal wavelet bases by the standard method oftranslations and dilations. It follows that if the starting fatherwavelet generates an orthogonal system, then, the corresponding waveletscreated by our wavelet generators, Equation (382), are also orthogonalsystems.

It is very easy to construct various delta-sequence-generated waveletsby applying examples given in Section II to the right hand side ofEquation (383).

Wavelets Generated by Difference Pairs

Another simple and efficient way of generating wavelets from deltasequences is to take the difference between two normalized elements in adelta sequenceψ_(α,β)(x)=φ_(α)−φ_(β)  (393)

EXAMPLE 1 Hermite Wavelets and the Mexican Hat Wavelet

In case of Hermite's delta sequence, Equation (368), we have$\begin{matrix}{{\psi_{n,n^{\prime}} = {{{\mathbb{e}}^{{- x^{2}}/2}{\sum\limits_{k = 0}^{n}\quad{( \frac{- 1}{4} )^{k}\frac{1}{\sqrt{2\pi}{k!}}{H_{2k}( \frac{x}{\sqrt{2}} )}}}} -}}\quad} & (394) \\{{\mathbb{e}}^{{- x^{2}}/2}{\sum\limits_{k = 0}^{n^{\prime}}\quad{( \frac{- 1}{4} )^{k}\frac{1}{\sqrt{2\pi}{k!}}\quad H_{2k}\quad( \frac{x}{\sqrt{2}} )}}} & \quad \\{= {{\mathbb{e}}^{{- x^{2}}/2}{\sum\limits_{k = n^{\prime}}^{n}\quad{( \frac{- 1}{4} )^{k}\frac{1}{\sqrt{2\pi}{k!}}{H_{2k}( \frac{x}{\sqrt{2}} )}}}}} & (395)\end{matrix}$This is a general expression for a family of nonorthogonal wavelets. Inparticular, if n=1′=0, we obtain $\begin{matrix}{{\psi_{1,0}(x)} = {\frac{1}{2\sqrt{2\pi}}( {1 - x^{2}} ){{\mathbb{e}}^{{- x^{2}}/2}.}}} & (396)\end{matrix}$This is, once again, the well-known Mexican Hat Wavelet [3]. The Hermitewavelets described in Equation (396) can easily obtained, within aconstant difference, by appropriately choosing n′=n=1 in Equation (394).

EXAMPLE 2 Shannon's Wavelet

We can use Dirichlet's continuous delta sequence as a father wavelet${\phi_{\alpha}(x)} = {\frac{1}{\pi}{\frac{\sin( {\alpha\quad x} )}{x}.}}$Then the corresponding mother wavelets generated by Equation (393) are$\begin{matrix}{{\psi_{\alpha,\beta}(x)} = {{{\frac{1}{\pi}\frac{\sin( {\alpha\quad x} )}{x}} - {\frac{1}{\pi}\frac{\sin( {\beta\quad x} )}{x}\quad{for}\quad\alpha}} \neq \beta \neq 0}} & (397)\end{matrix}$

This family includes the well-known Shannon's wavelet [3] as a specialcase $\begin{matrix}{{\psi_{{2\pi},\pi}(x)} = {\frac{1}{\pi\quad x}\lbrack {{\sin( {2\pi\quad x} )} - {\sin( {\pi\quad x} )}} \rbrack}} & (398)\end{matrix}$It is easy to check that this Shannon's wavelet generates an orthogonalsystem. Equation (398) is in contrast to the other family of Shannon'swavelets, Equation (392), produced using our wavelet generators.

EXAMPLE 3 Gauss' wavelets

It is noted that this procedure of generating wavelets is also verygeneral. For example a wavelet can be constructed by combining a pair offunctions from the Gauss' delta sequence, $\begin{matrix}{{\psi_{\alpha,\beta}(x)} = {{{\frac{1}{\sqrt{\pi}\alpha}{\mathbb{e}}^{{{- x^{2}}/2}\alpha^{2}}} - {\frac{1}{\sqrt{\pi}\beta}{\mathbb{e}}^{{{- x^{2}}/2}\beta^{2}}\quad{for}\quad\alpha}} \neq \beta \neq 0}} & (399)\end{matrix}$Note that this is not a special case of Example 1.

In the case where there is more than one delta parameter, thecorresponding wavelets can be generated as differences of cross terms.This will be discussed in detail in Section V.

DISTRIBUTED APPROXIMATING FUNCTIONALS

Definition 5. A function is said to be of the Schwartz class if it is aC^(∞) function and rapidly decaying, $\begin{matrix}{{{\sup\limits_{x \in R}{{x^{i}{\partial^{j}\quad{\psi_{\alpha}(x)}}}}} < \infty},} & (400)\end{matrix}$for α≠0 and all pairs of i and j. The set of all functions of theSchwartz class is denoted by S.

Definition 6. Distributed Approximating Functionals (DAFs) are familiesof functions which satisfy the following:

-   -   1. They are sequences of Schwartz class functions on the real        line R;    -   2. They are multiple parameter sequences and each parameter        independently leads to a sequence of functions which converges        to the delta distribution δ    -   3. They are delta sequences of the Dirichlet type.

Definition 7. A function on (−∞, ∞) is said to be of the DAF class if itis integrable over every bounded interval and is of at most polynomialgrowth asymptotically.

Fourier transforms of Schwartz class functions are still Schwartz classfunctions. DAFs are smooth and rapidly decaying functions in bothFourier space and ordinary space. We note that these properties arecrucial to the usefulness of DAFs in various numerical applications.DAFs are a multi-parameter system. The best computational efficiency canonly be obtained in certain regions of the parameter space. For HermiteDAFs, we call these well-tempered regions. (Note that well-tempered usedhere has nothing to do with the tempered distributions which arisenaturally in the theory of Fourier transforms.) By well-tempered, wemean that the DAF approximations to a DAF class function are of the samelevel of accuracy both on and off a grid or DAF approximations of thederivatives of the function have compatible accuracy to that of thefunction. By compatible accuracy, we mean that the accuracy decreasesless than a factor of two as the order of differentiation increases byone. (Note that pointwise differentiability is not required for the DAFclass fuinctions because generalized derivatives exist for the DAFconvolution of a DAF class function. Certainly generalized derivativesare, in general, not necessarily functions but are distributions. Thatis how DAFs can lead to distribution solutions to a partial differentialequations and why they turn out to be extremely powerful forapproximating functions which have discontinuities and evensingularities on a set of measure zero.) An alternative way ofunderstanding the well-tempered region of DAF parameter space is to viewit from the wavelet multiresolution analysis point of view. Essentially,there is an interplay of three factors: (i) the DAF expansion of order M(which determines a father wavelet), (ii) a DAF window size σ (whichdetermines the scale or correlations), (iii) the DAF centralfrequency˜1/Δ (which is equivalent to a dilation parameter). The bestcomputational efficiency is achieved when, for a given M, the DAF windowsize is proportional to the central frequency, σ∝Δ.

The last condition in our definition of DAFs reflects the fact that themost important DAFs we have discovered so far are delta sequences of theDirichlet type. From the point of view of approximation theory, deltasequences of the Dirichlet type are, in general, more rapidly ohvergentthan those of the positive type [39]. However, for providing unbiasedapproximations, the delta sequence of the Dirichlet type is applicableto a smaller class of functions than that of the positive type.Specifically, the function is required to be at lease Holder-continuousor C¹. This requirement is not needed for the DAF approximator becauseDAFs are of Schwartz class. An additional benefit of DAFs being ofDirichlet type is that DAFs are indeed “small waves” [2] and are readilyadmitted as father wavelets. Corresponding DAF mother wavelets can begenerated by the techniques described in the last section.

In parallel to the orthogonal wavelets and nonorthogonal wavelets, thereare two classes of DAFs: orthogonal DAFs which are generated byorthogonal basis expansion of the delta distribution and nonorthogonalDAFs which are created by methods of regularization. These are discussedin the following two subsections. It is noted that both orthogonal DAFsand nonorthogonal DAFs are frames, rather than orthogonal fatherwavelets.

A. DAFs Generated from Orthogonal Systems

One of the most important DAFs is the Hermite DAF (HDAF). It was firstintroduced in a discrete from [25] by Hoffman, Kouri and coworkers forquantum dynamics and then in a continuous form, by Kouri, Hoffman andcoworkers [26], as $\begin{matrix}{{{\delta_{HDAF}( { {x - x^{\prime}} \middle| M ,\sigma} )} = {\frac{1}{\sigma}{\exp( \frac{- ( {x - x^{\prime}} )^{2}}{2\sigma^{2}} )}{\sum\limits_{n = 0}^{M/2}{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2\pi}{n!}}{H_{2n}( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )}}}}},} & (401)\end{matrix}$where $H_{2n}( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )$are the usual even Hermite polynomials. While the Hermite expansion ofthe delta distribution, Equation (368), was discussed by Schwartz [35]and Korevaar [36], the Hermite DAF is very different, at least whereapplications are concerned, because it is explicitly a two-parameterdelta sequence. This has the consequence that the Hermite DAF _(δ)_(HDAF) _((x−x′|M,σ)) converges to the delta distribution either whenthe degree of the polynomial M goes to infinity $\begin{matrix}{{\lim\limits_{Marrow\infty}{\delta_{HDAF}( { {x - x^{\prime}} \middle| M ,\sigma} )}} = {\delta( {x - x^{\prime}} )}} & (402)\end{matrix}$independent of σ≠0 or in the limit σ−>∞, $\begin{matrix}{{\lim\limits_{\sigmaarrow 0}{\delta_{HDAF}( { {x - x^{\prime}} \middle| M ,\sigma} )}} = {\delta( {x - x^{\prime}} )}} & (403)\end{matrix}$independent of 0≦M<∞. Having two independent DAF parameters is crucialto its success in various numerical applications, because adjusting twoDAF parameters leads to tunable accuracy within well-tempered regions.Without the introduction of the dilation parameter σ in the Hermiteexpansion of Delta distribution, Equation (368) itself would have verylittle numerical utility. Well-tempered Hermite DAFs have been used fora variety of numerical applications, including among others, filteringand fitting experimental data and ab initio quantum mechanical potentialsurface values, padding data on two- and three- dimensional surfaces,resolving eigenvalues of a Hamiltonian solving linear and nonlinearPDEs, and signal processing. Some of these applications involveestimating the derivatives of a function known only on a finite,discrete grid, which are given in our DAF method in the sense of adistribution [25], $\begin{matrix}{{f^{(q)}(x)} = \langle {{\delta_{HDAF}^{(q)}( { {x - x^{\prime}} \middle| M ,\sigma} )},{f( x^{\prime} )}} \rangle} & (404) \\{\quad{{= {( {- 1} )^{q}\langle {{\delta_{HDAF}^{(q)}( { {x - x^{\prime}} \middle| M ,\sigma} )},{f^{(q)}( x^{\prime} )}} \rangle}},}} & (405)\end{matrix}$where δ_(DAF)^((q))(x − x^(′)|M, σ)is given by [25] $\begin{matrix}{{{\delta_{HDAF}^{(q)}( { {x - x^{\prime}} \middle| M ,\sigma} )} = {\frac{1}{\sigma^{q + 1}}\exp\quad( \frac{- ( {x - x^{\prime}} )^{2}}{2\sigma^{2}} ){\sum\limits_{n = 0}^{M/2}\quad{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2^{q + 1}\pi}{n!}}\quad H_{{2n} + q}\quad( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )}}}},} & (406)\end{matrix}$As stated earlier, the Hermite DAF is a Schwartz class function fornon-zero σ and the finite M, the right hand side of Equation (404)exists in a distribution sense of ƒ^((q)) is not well defined. Thissituation can occur in solving PDEs for which Herrmite DAFs can smoothout the discontinuities and “round off” singularities so as to providenumerical solutions (so called “weak solutions”). This is the case insolving the Fokker-Planck equation when the initial distributionfunction ƒ is a Dirac delta distribution. Hermite DAFs have successfullyestimated the first and second derivatives and integrated theFokker-Planck equation to an L₂ error of the order of 10⁻¹² [30].

The Fourier space representation of the Hermite DAF is given by [27]$\begin{matrix}{{{\hat{\delta}}_{HDAF}( { \xi \middle| M ,\sigma} )} = {{\exp( {- \frac{\xi^{2}\sigma^{2}}{2}} )}{\sum\limits_{n = 0}^{M/2}{\frac{\lbrack {( {\sigma\quad\xi} )^{2}/2} \rbrack^{n}}{n!}.}}}} & (407)\end{matrix}$

This has the important property that{circumflex over (δ)}_(HDAF)(0|M, σ)=1∀σ≠∞, M/2=1,2, . . .   (408)Hence, Hermite DAFs are readily admissible as father wavelets. Inparallel with Equations (402) and (403), it is easy to see that HermiteDAFs become a Fourier space all pass filter, i.e., {circumflex over(δ)}_(HDAF)(ξ|M,σ), whenever M→∞ or σ→0. For a given set of Hermite DAFparameters M≠∞,σ≠0,{circumflex over (δ)}_(HDAF)(ξ|M,σ) is an infinitelysmooth, exponentially decaying low pass filter which is extremely usefulfor signal analysis.

DAFs of Fejer type can be easily generated by using our Hermite DAFs$\begin{matrix}{{\delta_{{HDAF},{Fejér}}( { {x - x^{\prime}} \middle| M ,\sigma} )} = {\frac{2}{\sigma( {M + 2} )}{\exp( \frac{- ( {x - x^{\prime}} )^{2}}{2\quad\sigma^{2}} )}{\sum\limits_{f = 0}^{M/2}{\sum\limits_{n = 0}^{f}{( {- \frac{1}{4}} )^{n}\quad\frac{1}{\sqrt{2\quad\pi}{n!}}{{H_{2n}( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )}.}}}}}} & (409)\end{matrix}$A somewhat more general approach is to sum selectively $\begin{matrix}{ { {{\delta_{{HDAF},{Fejér}}( x } - x^{\prime}} \middle| M ,M^{\prime},\sigma} ) = {\frac{2}{\sigma( {M + M^{\prime}} )}\quad{\exp( \quad\frac{- ( {x - x^{\prime}} )^{2}}{2\quad\sigma^{2}} )}\quad{\sum\limits_{f = {M^{\prime}/2}}^{M/2}{\sum\limits_{n = 0}^{f}{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2\quad\pi}{n!}}{{H_{2n}( \frac{x - x^{\prime}}{\sqrt{2}\sigma} )}.}}}}}} & (410)\end{matrix}$We point out that since DAFs are multiparameter delta sequences,averaging over other DAF parameters will also generate DAFs of Fejertype. We shall not, however, give an exhaustive list of thesepossibilities here.

A particular, special case of our Hermite DAF (for M=2) has beenpreviously discussed by Monaghan and Gingold [47} in terms of a superGaussian kernel, $\begin{matrix}{{W( {x,\sigma} )} = {\frac{1}{\sqrt{2\pi}\sigma}{\exp( {- \frac{x^{2}}{2\sigma^{2}}} )}{( {\frac{3}{2} - \frac{x^{2}}{\sigma^{2}}} ).}}} & (411)\end{matrix}$(Note that this is different from the Mexican hat wavelet.) Theseauthors noted that this super Gaussian kernel gives interpolation errorof O(σ⁴) in numerical integrations and performs better than the Gaussiankernel.

More general DAFs based on orthogonal basis expansions can beconstructed in two ways, as described in Ref. [28]. Essentially, for agiven positively defined weight function w in a domain (a, b), it ispossible, by using the standard three term recurrence method, togenerate a set of polynomials P_(n) which is orthonormal under the innerproduct with respect to the weight w. This results in very generalorthonormal L²(a,b) bases. All classical polynomial systems, such asHermite, Legendre, Jacobi, Chebyshev, etc. are special cases of thisapproach. This procedure of generating arbitrary orthonormal systems iswell-known and has been employed by Shizgal [48] for constructingvarious reproducing kernels, which, when limited to the classical weightfunction are coincident with the discrete variable representation (DVR)[49}. If the weight function of a general orthonormal basis is chosen tobe of the Schwartz class a general DAF can then be constructed byexpanding the delta distribution in the domain (a,b). The resulting DAFis an approximate identity kernel and is very similar to the reproducingkernel of Shizgal for a common weight w. However, there are importantdifferences in the philosophies of the two approaches. The grid pointsin the Shizgal's method are always restricted to the whole set of nodesof the polynomial of highest degree occurring in the expansion. As aresult, both the function and its derivatives are approximated in aglobal manner, necessarily involving all grid points in the domain(a,b). A major advantage in the DAF approach is that there is norestriction on grid selection. It turns out that because of the natureof the Schwartz class DAFs, numerically, the DAF approximation to afunction and its derivative at each point x is effectively a localizedone, with contributions coming only from near neighbor points, x′. Therange of the neighborhood is controlled by the DAF window parameter, σin case of Hermite DAFs. The functional relation determining thecontributions of neighboring grid points is given by a pointwise basisset expansion of the delta distribution and controlled by anotherindependent DAF parameter. Therefore, the DAF approach is a pointwisespectral method and its matix representation is banded. It is well-knownthat, in general, global methods are more accurate than local methods,while local methods are flexible for handling complex boundaries andgeometries. Therefore, linear systems with simple boundary conditionsare preferably solved by global methods, while nonlinear systems withcomplex boundaries are usually solved by local methods. However, in awide range of numerical applications, such as long-term weatherpredicting, describing shock waves in compressible gas flow, or vortexsheets in high Reynolds number incompressible fluid flow, and studyingcritical points of Bose-Einstein condensations, one deals with nonlinearPDEs possessing singularities or phase space homoclinic orbits [11,12].The accuracy of approximations to derivatives becomes particularlyimportant because homoclinic orbit crossing can induce numerical chaos[12]. Hence, it is desirable to have an approach that has global methodaccuracy and local method flexibility for treating the above mentionedhighly demanding systems. Spectral element approaches [50], whichcombine the spectral method with the finite element method, have beenstudied for this purpose. A general difficulty in this approach is theinterfacial matching between various spectral subdomains. This leads toa great reduction in accuracy. We find that DAFs provide a robustalternative approach for solving such problems.

A second way to construct general DAFs is based on the variationalprinciple also described in Ref. [28]. A cost function λ is defined as$\begin{matrix}{{\lambda( {\{ \alpha_{j} \},x} )} \equiv {\sum\limits_{k}{{w( {x_{k} - x} )}{{{f( x_{k} )} - {\sum\limits_{j}{{\alpha_{j}(x)}{\xi_{j}( {x_{k} - x} )}}}}}^{2}}}} & (412)\end{matrix}$where w is a positive weight function, {ξ_(j)} is a set of functions and{α_(j)} is a set of expansion coefficients to be determined by localminimization of λ({α_(j)}; x). It turns out that for an arbitrary localintegrable function ƒ, the set of coefficients {α_(j)} that minimizesthe λ({α_(j)}; x) is the same as that which provides an orthogonal basisexpansion of the delta distribution under the same weight w. We referthe reader to Ref. [28] for more details. It is noted that in thisapproach, each point can have its own basis set, independent of thebasis sets used at neighboring points. This is equivalent to using adifferent basis for expanding at each grid point. Since this result isvery general, we refer to it as the “DAF variational principle.” Thisapproach has been shown to be extremely powerfuil in molecular potentialsurface fitting [51], as well as in a variety of signal processingapplications. Similar approaches exist in the mathematical literature;for example, moving least squares have been discussed by many authors[52].

Orthogonal basis set expansions of the delta distribution have beendescribed in very general form in Ref. [28]. A historical summary oforthogonal basis-derived DAFs can be found in the reference section ofRef. [28].

DAFs Generated by Nonorthogonal Systems

-   -   1. Fourier Space Regularization

Definition 8. A tempered distribution on R is a linear mapping T:S→Rsuch that for some positive integer N and a constant C $\begin{matrix}{{{T\lbrack\phi\rbrack}} \leq {C\quad{\sum\limits_{{m + n} \leq N}{\sup{{x^{m}{\partial^{n}{\phi(x)}}}}\quad{\forall{\phi \in S}}}}}} & (413)\end{matrix}$

The space of all tempered distributions on R is denoted as S′. Theexamples of tempered distributions are all polynomials and all L¹, L²functions and even the periodic delta distribution Σ_(k=−∞)^(∞)δ(x=2πk). However, neither Σ_(k=−∞) ^(∞)δ(x=2πk). However, neitherΣ_(k=−∞) ^(∞)δ^((k))(x−2πk) nor e^(|x|)are tempered.

It is obvious that ∫Ŵ_(σ)(ξ)dξ=1 for all σ. We construct the ShannonGabor wavelet-DAF as{circumflex over (δ)}_(SGWD)(ξ|σ,η)={circumflex over (T)}*ŵ_(σ)  (417)This is a smoothed function, which has the property that{circumflex over (δ)}_(SGWD)(0|σ,η)=1  (418)The function ŵ₉₄(ξ) is a regularizer and{circumflex over (δ)}_(SGWD)(ξ|σ,η)Δ{circumflex over (T)}  (419)as σ→∞. We thus recover exactly the ideal low pass filter functionx_([−η,η]) in the limit of our regularization. Moreover, for a given σ,$\begin{matrix}{{\lim\limits_{\etaarrow\infty}{{\hat{\delta}}_{SGWD}( { \xi \middle| \sigma ,\eta} )}} = {1\quad{\forall{\xi \in R}}}} & (420)\end{matrix}$as required for DAFs.

The inverse Fourier transforin gives the Shannon Gabor wavelet-DAF [33]$\begin{matrix}{{\delta_{SGWD}( { x \middle| \sigma ,\eta} )} = {{{F^{- 1}\lbrack {{\hat{\delta}}_{SGWD}( { x \middle| \sigma ,\eta} )} \rbrack}(x)} = {{2\pi\quad T\quad\omega} = {\frac{1}{\pi}\frac{\sin( {\eta\quad x} )}{x}{\mathbb{e}}^{{{- x^{2}}/2}\sigma^{2}}}}}} & (421)\end{matrix}$This is still a two-parameter DAF such that $\begin{matrix}\lbrack {{{\lim\limits_{\etaarrow\infty}{{\delta_{SGWD}( { x \middle| \sigma ,\eta} )} \quad \rbrack}} = {{\delta(x)}\quad{\forall\sigma}}},{x \in R}}  & (422) \\\lbrack {{{\lim\limits_{\sigmaarrow\infty}{{\delta_{SGWD}( { x \middle| \sigma ,\eta} )} \quad \rbrack}} = {{\delta(x)}\quad{\forall\eta}}},{x \in R}}  & (423)\end{matrix}$

The Shannon Gabor wavelet-DAF can be regarded as a Gaussian regularizedDirichlet continuous delta sequence in the coordinate representation.

In numerical applications we chooser η={fraction (π/Δ)}(Δ=x_(n)−x_(n−1))for a discrete version of Shannon Gabor wavelet-DAF $\begin{matrix}{ { {{\delta_{SGWD}( x } - x_{n}} \middle| \sigma ,\Delta} ) = {\frac{1}{\Delta}\frac{\sin\lbrack {\frac{x}{\Delta}( {x - x_{n}} )} \rbrack}{\frac{x}{\Delta}( {x - x_{n}} )}{\mathbb{e}}^{{{- {({x - x_{n}})}^{2}}/2}\sigma^{2}}}} & (424)\end{matrix}$This form has the interpolating property.

EXAMPLE 2 Generalized de la Vallee Poussin DAF

We choose {circumflex over (T)} as the following $\begin{matrix}{{{\hat{T}}_{\eta,\lambda}(\xi)} = \{ \begin{matrix}1 & {{\xi } \leq \eta} \\{{\lambda\quad\eta} - {\xi\quad\eta}} & {\eta < {\xi } \leq {\lambda\quad\eta}} \\0 & {otherwise}\end{matrix} } & (425)\end{matrix}$where η≧0,λ>1. Let {circumflex over (δ)}_(DAF)(ξ|σ,η,λ)={circumflex over(T)}_(η,λ)*{circumflex over (ω)}_(σ). Then we can construct acorresponding DAF by $\begin{matrix}{{\delta_{DAF}( {x,\eta,\lambda,\sigma} )} = {{F^{- 1}\lbrack {\hat{\delta}*{\hat{\omega}}_{\sigma}} \rbrack} = {{2\quad\pi\quad T\quad\omega} = {\frac{1}{x}\frac{{\cos( {\eta\quad x} )} - {\cos( {\lambda\quad\eta\quad x} )}}{( {\lambda - 1} )\eta\quad x^{2}}{\mathbb{e}}^{{{- x^{2}}/2}\sigma^{2}}}}}} & (426)\end{matrix}$This DAF reduces to the Shannon Gabor wavelet-DAF, Equation (421), inthe limit of λ→1 and to the de la Vallee Poussin DAF [32] when λ=2. Thelatter has been tested for some numerical applications, includingsolving PDEs [32].

EXAMPLE 3 Arbitrary Schwartz Class Filters

As discussed in Ref. [53], to design a general, smoothed filter, whichcan be low pass, high pass, band pass or band stop, we choose a Schwartzclass function {circumflex over (ω)}∈S which satisfies∫{circumflex over (ω)}(ξ)dξ=1   (427)and {circumflex over (ω)}_(σ)(ξ)=σ{circumflex over (ω)}(σξ). Its Fourierinverse satisfies0≦ω_(σ)(x)≦1 ∀x∈R.  (428)

Equation (84) implies that ω_(σ)(0)=1. Let {circumflex over (T)}_(α) ₁_(,α) ₂ _(, . . .) (ξ) be a piecewise smooth function characterized by aset of parameters {α₁} and satisfying0≦{circumflex over (T)}_(α) ₁ _(,α) ₂ _(, . . .) (ξ)≦1 ∀ξ∈R.  (429)Then we have a generalized Schwartz class filterδ_(DAF)(x)=F⁻¹[{circumflex over (T)}*{circumflex over(ω)}](x)=2πT(x)ω(x)  (430)This obviously includes Example 1 and Example 2 as special cases.

Another interesting special case is given by choosing $\begin{matrix}{{{\hat{T}}_{\eta,\lambda}(\xi)} = \{ {\begin{matrix}1 & {\beta \leq {\xi } \leq \alpha} \\0 & {otherwise}\end{matrix}\begin{matrix}{\alpha \geq \beta > 0} \\\quad\end{matrix}} } & (431)\end{matrix}$We then have a smoothed high pass filter $\begin{matrix}{{\delta_{HPF}( { x \middle| \sigma ,\alpha,\beta} )} = {{\frac{1}{\pi\quad x}\lbrack {{\sin( {\alpha\quad x} )} - {\sin( {\beta\quad x} )}} \rbrack}{\mathbb{e}}^{- \frac{x^{2}}{2\sigma^{2}}}}} & (432)\end{matrix}$We note that this is equivalent to a DAF-wavelet. Equation (432) reducesto the Shannon Gabor wavelet-DAF when β=0.

The method presented in this section is very general for theconstruction of arbitrary smooth filters and DAFs. (The desired degreeof smoothness is obtained by appropriately choosing the regularizer.) Werefer the reader to Ref. [53] for more details.

Interpolating Formulae and DAFs

Since a Fourier space convolution leads to a coordinate space productfor a certain class of functions, it is clear that regularization inFourier space by a Schwartz class function is equivalent to a coordinatespace regularization, by a Schwartz class function. Therefore,coordinate space S-function regularization of commonly usedinterpolating kernels leads to smoothed kernels, or DAFs. The mostwell-known such kernel is the Lagrange interpolating formulae$\begin{matrix}{{L_{n}( {f,x} )} = {\sum\limits_{i}^{n}\quad{{l_{i}(x)}{f( x_{i} )}\quad{\forall{x_{i} \in ( {a,b} )}}}}} & (433)\end{matrix}$where l_(i)(x) is the well-known Lagrange kernel given by$\begin{matrix}{{{l_{i}(x)} = {\prod\limits_{{j = {- M}},{j \neq i}}^{M}\quad{\frac{x - x_{j}}{x_{i} - x_{j}}\quad{\forall x_{i}}}}},{x_{j} \in ( {a,b} )}} & (434)\end{matrix}$(Note that by using the summation convention, Equation (433), allkernels and DAFs presented in this and the next subsections differ fromthose in the previous sections by a factor of {fraction (1/Δ)}, where Δis the grid spacing.) L agrange interpolating formulae yield polynomialapproximations to ƒ(x) when its values are given on a set of n nodepoints {x_(i)}. If the nodes are chosen to be equidistant,x_(i=−)1+2(i−1)/n−1, i=1,2, . . . ,n, then integration gives the famousNewton-Cotes quadrature formulae. We have constructed Lagrange DAFs byintroducing a Gaussian weight to the Lagrange interpolating formulae$\begin{matrix}{{\delta_{LDAF}( {x - x_{i}} \middle| \sigma )} = {\prod\limits_{{j = {- M}},{j \neq i}}^{M}\quad{\frac{x - x_{j}}{x_{i} - x_{j}}{\mathbb{e}}^{- \frac{{({x - x^{\prime}})}^{2}}{2\sigma^{2}}}}}} & (435)\end{matrix}$The most important difference between our Lagrange DAF and the Lagrangeinterpolating formulae is the way in which nodal points x_(i), x_(j) arechosen. The nodes x_(j) of the polynomials are always chosen around eachcentral point x_(i)∈(a,b). Therefore when x_(i) is close to the boundarya or b, some nodal points x_(j) will fall outside of (a, b). This causesour Lagrange DAF approximation to have translational symmetry and beasymptotically unbiased in the whole computational domain (a, b). Thisway of choosing grid points is due to the fact that DAFs are essentiallya (−∞,∞) domain method rather than a bounded (a, b) domain method. Thisreflects the nature of the Schwartz space weight function. However,being a (−∞,∞) domain method does not mean we have to use the nodes at±∞, and it also does not mean that we cannot use it for finite domain(a, b) calculations. On the contrary, the Lagrange DAF is more flexiblefor arbitrary finite domain computations, because the S-space weightfunction makes our Lagrange DAF kernel an effectively localized one. TheLagrange DAF matrix is effectively banded. In principle, on a portion(2M +1) of all the grid points which are closest to x_(i) must be usedin constructing an approximation to the kernel δ_(LDAF)(x−x_(i)|σ). Ifthe nodes are chosen to be equally spaced, then we only need tocalculate δ_(LDAF)(x−x_(i)|σ) once for all {x_(i)} in (a,b).

More general interpolating DAFs can be constructed by using more generalLagrange interpolations. Let L_(i)(x), i=1,2, . . . , n be a set offunctions satisfyingL _(i)(x _(j))=δ_(i,j)   (436)As a result, the generalized Lagrange formulae $\begin{matrix}{{L_{n}( {f,x} )} = {\sum\limits_{i = 1}^{n}\quad{{f( x_{i} )}{L_{i}(x)}}}} & (437)\end{matrix}$will have the interpolating property, so thatL _(n)(f,x _(m))=ƒ(x _(m)), m=1,2, . . . , nproviding that the x_(i) are n distinct nodes. A very efficient way forobtaining L_(i)(x) of the Lagrange type is to choose a nodal functionμ_(n)(x) and a basis function v_(i)(x) defined in an open intervalaround every node, and which have at least the first derivative at everynode, satisfyingμ_(n)(x _(i))=0, 0<|μ¹ _(n)(x _(i))|<∞ ∀i=1,2, . . . , nv _(i)(x _(i))=0, 0<|v ¹ _(i)(x _(i))|<∞ ∀i=1,2, . . . , nv _(i)(x _(m))≠0 ∀i≠m  (438)The generalized Lagrange kernels are then obtained by making$\begin{matrix}{{{L_{i}(x)} = \frac{{\mu_{n}(x)}{v_{i}^{\prime}( x_{i} )}}{{\mu_{n}^{\prime}( x_{i} )}{v_{i}(x)}}},\quad{i = 1},2,\ldots\quad,{n.}} & (439)\end{matrix}$This satisfies Equation (436) and consequently Equation (437). Acorresponding generalized Lagrange DAF can be constructed as$\begin{matrix}{{\delta_{GLDAF} = {\frac{{\mu_{n}(x)}{v_{i}^{\prime}( x_{i} )}}{{\mu_{n}^{\prime}( x_{i} )}{v_{i}(x)}}{w_{i}(x)}}},} & (440)\end{matrix}$where w_(i)(x)∈S is a Schwartz class function such that0≦ω_(i)(x)≦1 and ω_(i)(x _(i))=1 ∀x∈R  (441)Equation (440) is a general expression for a large class ofinterpolating DAFs.

EXAMPLE 1 Lagrange Interpolation

The Lagrange kernel, Equation (434) is obtained by choosing$\quad\begin{matrix}\begin{matrix}{{\mu_{n}(x)} = {\prod\limits_{j = 1}^{n}\quad{( {x - x_{j}} )\quad{and}}}} \\{{{v_{i}(x)} = {x - x_{i}}},\quad{i = 1},2,\ldots\quad,n,\quad{then}} \\{{L_{i}(x)} = {\prod\limits_{{j = 1},{j \neq i}}^{n}\quad{\frac{x - x_{j}}{x_{i} - x_{j}}.}}}\end{matrix} & (442)\end{matrix}$The corresponding DAF given in Equation (435).

EXAMPLE 2 Cardinal Interpolating

Considering a set of equally spaced nodes {x_(i)}={0,±Δ,±2Δ, . . . ±∞}and

 μ_(n)(x)=sin({fraction (π/Δ)}x) and

v _(i)(x)=x−iΔ, i=0,±1,±2, . . . ,±∞; then $\begin{matrix}{{{L_{i}(x)} = {{\frac{( {- 1} )^{i}{\sin( {\frac{\pi}{\Delta}x} )}}{\frac{\pi}{\Delta}( {x - {i\quad\Delta}} )}\quad i} = 0}},{\pm 1},{\pm 2},\cdots,{\pm {\infty.}}} & (443)\end{matrix}$This is a different form of the well-known sinc interpolating formulaeand a corresponding DAF can be constructed as $\begin{matrix}{{{\delta_{SGWD}( {x - x_{i}} \middle| \sigma )} = {{\frac{( {- 1} )^{i}{\sin( {\frac{\pi}{\Delta}x} )}}{\frac{\pi}{\Delta}( {x - {i\quad\Delta}} )}{\mathbb{e}}^{- \frac{{({x - {i\quad\Delta}})}^{2}}{2\sigma^{2}}}\quad i} = 0}},{\pm \Delta},{{\pm 2}\Delta},\cdots\quad,{\pm {M.}}} & (444)\end{matrix}$This is a different form of the Shannon-Gabor wavelet-DAF. As in theLagrange DAF case, we only include 2M+1 nodes in the calculation. Laterin this subsection, we show that this is also a special case of ageneralized Lagrange DAF (derived for evenly distributed nodes).

EXAMPLE 3 Trigonometric Interpolation

Let |x_(i)|<1 and $\begin{matrix}\begin{matrix}{{{v_{i}(x)} = {{{\sin( {\frac{\pi}{2}( {x - x_{i}} )} )}\quad x_{i}} = 1}},2,\cdots,n} \\{{{\mu_{n}(x)} = {\prod\limits_{l}^{n}\quad{v_{i}(x)}}};\quad{then}} \\{{L_{i}(x)} = \frac{\pi\quad{\mu_{n}(x)}}{2{\mu_{n}^{\prime}( x_{i} )}{\sin\lbrack {\frac{\pi}{2}( {x - x_{i}} )} \rbrack}}}\end{matrix} & (445)\end{matrix}$is an interpolating kernel. Obviously it has the general form of aLagrange interpolation. We construct a corresponding DAF as$\begin{matrix}{{\delta_{SINDAF}( {x - x_{i}} \middle| \sigma )} = {\prod\limits_{{j = {- M}},{j \neq i}}^{M}\quad{\frac{\sin( {\frac{\pi}{2}( {x - x_{j}} )} )}{\sin( {\frac{\pi}{2}( {x_{i} - x_{j}} )} )}{{\mathbb{e}}^{- \frac{{({x - x_{i}})}^{2}}{2\sigma^{2}}}.}}}} & (446)\end{matrix}$Equation (445) is simplified for the case of equally spaced nodesx_(i)=−1+2i/(2n+1), i=0,1,2, . . . 2n to give $\begin{matrix}{{{L_{i}(x)} = \frac{\sin( {\frac{{2\quad n} + 1}{2}\quad\pi\quad( {x - \frac{{2\quad n} - {2\quad i} + 1}{{2\quad n} + 1}} )} )}{( {{2\quad n} + 1} )\quad{\sin( {\frac{\pi}{2}\quad( {x - \frac{{2\quad n} - {2\quad i} + 1}{{2\quad n} + 1}} )} )}}},\quad{i = 1},2,\ldots\quad,n} & (447)\end{matrix}$This is recognized as the Dirichlet kernel arising in Fourier analysis.The corresponding DAF can be written as $\begin{matrix}{{\delta_{DGWPD}( {{x - x_{i}}❘\sigma} )} = {\frac{\sin\quad\frac{\pi}{\Delta}\quad( {x - x_{i}} )}{( {{2\quad n} + 1} )\quad\sin\quad\frac{\pi}{\Delta}\quad\frac{( {x - x_{i}} )}{{2\quad n} + 1}}\quad{\mathbb{e}}^{- \frac{{({x - x_{i}})}^{2}}{2\quad\sigma^{2}}}}} & (448)\end{matrix}$This is our Dirichlet-Gabor-wavelet-packet-DAF, which was introducedwhen the connection between DAFs and wavelets was first made [32]. Aspointed out in Ref. [62], the Shannon-Gabor-wavelet-DAF can be regardeda special case of Equation (448) because $\quad\begin{matrix}{{\delta_{SGWD}\quad( {{x - x_{i}}❘\sigma} )} = {{\underset{narrow\infty}{\lim\quad}\quad\frac{\sin\quad\frac{\pi}{\Delta}\quad( {x - x_{i}} )}{( {{2\quad n} + 1} )\quad\sin\quad\frac{\pi}{\Delta}\quad\frac{( {x - x_{i}} )}{{2\quad n} + 1}}\quad{\mathbb{e}}^{- \frac{{({x - x_{i}})}^{2}}{2\quad\sigma^{2}}}} = {\frac{\sin\quad\frac{\pi}{\Delta}\quad( {x - x_{i}} )}{\frac{\pi}{\Delta}\quad( {x - x_{i}} )}\quad{\mathbb{e}}^{- \frac{{({x - x_{i}})}^{2}}{2\quad\sigma^{2}}}}}} & (449)\end{matrix}$All of the examples in this subsection have been tested by numericalapplications. They all have similar levels of numerical accuracy forinterpolating functions and for numerically solving linear and nonlinearPDEs. It is noted that the prescription given in Equations (438) and(439) is very general and efficient for constructing interpolationkernels. One can easily write down as many more interpolationexpressions as are desired. As a consequence, one can generate variousgeneralized Lagrange DAFs by using Equation (440) with S-class weightssatisfying Equation (441). It is beyond the scope of this paper to givean exhaustive list of all possibilities. (It is also true that there areinfinitely many orthogonal series expansion DAFs as noted in Ref. [28].)

The connection between interpolation DAF and the “DAF variationalpurposes. This idea led us to introduce Gaussian regularized Lagrangesampling in Ref. [57].

EXAMPLE 1 Shannon's Sampling

Shannon's sampling theorem is one of the most important mathematicalfoundations for information theory and various engineering fields. Iteven influences statistical mechanics and serves as an importantmotivation for wavelet development. Shannon's sampling theorem addressesthe issue of constructing or recovering a continuous function f(x) onthe real line x∈R from an infinite, discrete set of known values{f(x_(n))} $\begin{matrix}{{{f(x)} = {\sum\limits_{n = {- \infty}}^{\infty}{{f( x_{n} )}\quad\frac{\sin\quad{\eta( {x - x_{n}} )}}{\eta\quad( {x - x_{n}} )}}}},\quad{f \in B_{\eta}^{2}},\quad{x_{n} = \frac{n\quad\pi}{\eta}}} & (450)\end{matrix}$where Bx/x is the Paley-Wiener space of functions band limited to η,i.e., their momentum representations are identically zero outside theband ηh. This theorem provides connections between experimentalmeasurement (which is discrete in nature) and theoretical predictions(which is continuous in nature). The reproducing kernel $\begin{matrix}{\frac{\sin\quad{\eta( {x - y} )}}{\eta\quad( {x - y} )} = {\sum\limits_{n = {- \infty}}^{\infty}{\frac{\sin\quad{\eta( {x - x_{n}} )}}{\eta\quad( {x - x_{n}} )}\quad\frac{\sin\quad{\eta( {y - x_{n}} )}}{\eta\quad( {y - x_{n}} )}}}} & (451)\end{matrix}$is related to Whittaker's cardinal series $\begin{matrix}{\frac{\sin\quad{\eta( {x - x_{n}} )}}{\eta\quad( {x - x_{n}} )} = \frac{( {- 1} )^{n}\quad\sin\quad\eta\quad x}{{\eta\quad x} - {n\quad\pi}}} & (452)\end{matrix}$and to the Dirichlet continuous delta sequence, Equation (363), forappropriate choice of η. It is also known for generating an orthonormalbasis for the reproducing kernel Hilbert space Bx/x. By setting η=π, oneobtains the well-known Shannon's father wavelet, $\begin{matrix}{{\phi\quad(x)} = \frac{\sin\quad\pi\quad x}{\pi\quad x}} & (453)\end{matrix}$with the Fourier transform φ(x)=X_((−1/2, 1/2)) And the mother wavelet${\psi(x)} = \frac{{\sin\quad( {2\quad\pi\quad x} )} - {\sin( {\pi\quad x} )}}{\pi\quad x}$as discussed in Section III.

Shannon's wavelets are not efficient from a computational point of viewbecause of their slow decay as x becomes large. This is implied from theideal lowpass property. Mathematical sampling theory emphasizes the factthat expression Equation (450) is exact. However, in the real world,since one cannot actually use infinitely many sampling points, the“exactness” in Equation (450) is not physically realizable. Moreover,there is a well-known paradox [58] regarding the notion of band-limitedsignals. The usual definition implies that a band-limited signal is anentire function, whose Fourier transform has compact support. However,an entire function cannot have compact support unless it is identicallyzero in the entire domain. Therefore, it cannot be both band-limited andtime-limited, unless it is identically zero. This is in contrast to thefact that physically realizable states are well-behaved Schwartz-spacefunctions, which are effectively, but not exactly, both bandlimited andtime-limited [59]. The bandwidths in the frequency and time domains areactually related by the Heisenberg uncertainty principle. To construct apractically useful sampling formula which does not demand infinitelymany sampling points while providing as high accuracy as desired for anapplication, we employ the regularization procedure discussed above andchoose $\begin{matrix}{{w_{\sigma}(x)} = {\mathbb{e}}^{- \frac{x^{2}}{\quad^{2\quad\sigma^{2}}}}} & (454)\end{matrix}$to smooth out the Gibbs' oscillations and consequently reduce truncationerrors. This leads us to define an interesting Shannon-Gabor fatherwavelet (SGFW) as $\begin{matrix}{{\Phi(x)} = {\frac{\sin\quad\pi\quad x}{\pi\quad x}{\mathbb{e}}^{- \frac{x^{2}}{2\sigma^{2}}}}} & (455)\end{matrix}$Note that the Shannon-Gabor father wavelet is different from either theShannon father wavelet φ or the Gabor wavelet e^(−x) ² ^(/2σ) ² cos(αx).It is interesting to examine two limiting cases of the Shannon-Gaborfather wavelet: $\begin{matrix}{{\lim\limits_{\sigma->\infty}\quad{\Phi(x)}} = \frac{\sin\quad\pi\quad x}{\pi\quad x}} & (456)\end{matrix}$and $\begin{matrix}{{\lim\limits_{\sigma->0}{\frac{1}{2\quad{\pi\sigma}}{\Phi(x)}}} = {{\frac{\sin\quad\pi\quad x}{\pi\quad x}\quad{\delta(x)}} \equiv {\delta(x)}}} & (457)\end{matrix}$Shannon's father wavelet is recovered in the first limit (provided W→∞).The second limit, Equation (457), indicates that the Shannon-Gaborfather wavelet is a delta sequence. Using this fact, we can construct aninterpolating Shannon-Gabor distributed approximating functional (DAF)[33] $\begin{matrix}{{\delta_{DAF}( {{x - x_{n}}❘\sigma} )} = {\frac{\sin\quad\frac{\pi}{\Delta}( {x - x_{n}} )}{\frac{\pi}{\Delta}( {x - x_{n}} )}\quad{\mathbb{e}}^{- \frac{{({x - x_{n}})}^{2}}{2\sigma^{2}}}}} & (458)\end{matrix}$The advantage of this expression over Shannon's reproducing kernel,Equation (451), is that Equation (458) is effectively banded. The best σvalues form numerical purposes are determined by the dilationfactor˜Δ(grid spacing). For a given Δ, there is a wide range of σ's thatdeliver excellent numerical results. This Shannon-Gabor wavelet-DAF hasbeen tested on many numerical applications and is extremely accurate androbust for numerical solutions of linear and nonlinear partialdifferential equations.

EXAMPLE 2 Generalized Lagrange Sampling

Perhaps the most general sampling theorem is due to Paley and Wiener[60]. For an L₂ function ƒ which is band-limited to η, its value ƒ(x) atan arbitrary point x can be exactly recovered from an infinite sent of(not necessarily uniform) discrete “sampling points” [x_(k)].$\begin{matrix}{{{\sup\limits_{k \in Z}{{x_{k} - \frac{k\quad\pi}{\eta}}}} < \frac{\pi}{4\quad\eta}},} & (459)\end{matrix}$(Note that this implies a value of Δ_(k), one then constructs thefollowing Lagrange-type interpolating series $\begin{matrix}{{{f(x)} = {\sum\limits_{- \infty}^{\infty}{{f( x_{k} )}{S_{k}(x)}}}},\quad( {x \in R} )} & (460)\end{matrix}$where $\begin{matrix}{{S_{k}(x)} = \frac{G(x)}{{G^{\prime}( x_{k} )}\quad( {x - x_{k}} )}} & (461)\end{matrix}$is a Lagrange-type sampling function. Here G(x) is an entire functiongiven by $\begin{matrix}{{{G(x)} = {( {x - x_{0}} ){\prod\limits_{k = 1}^{\infty}\quad{( {1 - \frac{x}{x_{k}}} )\quad( {1 - \frac{x}{x_{- k}}} )}}}},} & (462)\end{matrix}$and G′ denotes the derivation of G. Equation (460) is called the Paleyand Wiener sampling theorem in the mathematical literature and can beregarded as a generalization of the classical Lagrange interpolationformula to the real line (R) for functions of the exponential type.Unlike the classical Lagrange interpolation formula, Equation (460)contains infinitely many terms, and we stress that it yields the exactƒ(x) for all real x. Thus, the interesting point is that the informationof a continuous function (containing a compact set of frequencies) onthe real line (R) can be entirely embedded in an infinite, but discreteirregularly placed set of sampling points (grid points). ConditionEquation (459) is the best one can have [60]. There will be an aliasingerror if the grid mesh is larger than is allowed by condition Equation(459) or if the function ƒ(x) is not band-limited to η. The majordisadvantage of Equation (460) is that it converges slowly. In practice,neither computational nor experimental data can ever be obtained at aninfinite set of discrete sampling points. As noted above, from amathematical point of view, a band-limited (i.e., compact support inFourier space) function cannot have compact support in the coordinaterepresentation unless it is identically zero. From a physical point ofview, physical measurements cannot be conducted for an infiniteduration, therefore physically realizable states are the Schwartz-classfunctions [59], which can be treated as effectively band-limited in boththe momentum and coordinate representations. This suggests that one cantruncate Equation (460) and still obtain reasonable results. A simpleway of achieving this is to introduce a weight function w_(k)(x) intothe right hand side of Equation (460). (This approach can be maderigorous by introducing the regularization in the momentum space, asdiscussed above.) This leads to the approximate equation $\begin{matrix}{{f(x)} \approx {\sum\limits_{- \infty}^{\infty}\quad{{f( x_{k} )}{S_{k}(x)}{w_{k}(x)}}}} & (463)\end{matrix}$A particularly robust weight function on the real line R is the Gaussian$\begin{matrix}{{{w_{k}(x)} = {\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\quad\sigma_{k}^{2}}}},\quad( {0 < \sigma < \infty} )} & (464)\end{matrix}$Note that the approximate Equation (463) becomes exact in the limit thatσ_(k) approaches infinity. Moreover, as σ_(k) tends to zeroS^(k)(x)w_(k)(x) behaves like a “semi-continuous” Dirac delta function,$\begin{matrix}{{{\lim\limits_{\sigma_{k}arrow 0^{+}}{\frac{1}{\sqrt{2\pi}\sigma}\quad\frac{G(x)}{{G^{\prime}( x_{k} )}( {x - x_{k}} )}\quad{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\quad\sigma_{k}^{2}}}}} = {\frac{G(x)}{{G^{\prime}( x_{k} )}( {x - x_{k}} )}\quad{{\delta( {x - x_{k}} )}.}}}\quad} & (465)\end{matrix}$This is effectively a delta function because $\begin{matrix}{{{\lim\limits_{xarrow x_{k}}\frac{G(x)}{{G^{\prime}( x_{k} )}( {x - x_{k}} )}} = 1.}\quad} & (466)\end{matrix}$Therefore the kernel S^(k)(x)w_(k)(x) of Equation (463) can beapproximated as an integral over x_(k), then one also obtains exactresults as σ_(k)→0⁺. For a finite set of sampling points {x_(k)}^(M)k=1which are distributed in the nearest neighbor region of point x_(k), wehave the following LDAF expression [29] $\begin{matrix}{{\delta_{LDAF}( {{{x - x_{k}}❘M},\sigma_{k}} )} = {\prod\limits_{i \neq k}^{M}\quad{\frac{( {x - x_{i}} )}{x_{k} - x_{i}}\quad{{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\quad\sigma_{k}^{2}}}.}}}} & (467)\end{matrix}$The Lagrange sampling theorem is very general and it includes Shannon'ssampling as a special case of x_(k)=kΔ=−x_(−k). Then it is seen thatEquation (462) becomes $\begin{matrix}{{G(x)} = {x{\prod\limits_{{k = {- \infty}},{k \neq 0}}^{\infty}\quad( {1 - \frac{x}{k\quad\Delta}} )}}} & {{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}(468)} \\{= {x{\prod\limits_{k = 1}^{\infty}\quad( {1 - \frac{x^{2}}{k^{2}\quad\Delta^{2}}} )}}} & {(469)} \\{= {\Delta{\frac{\sin\quad\frac{\pi}{\Delta}\quad x}{\pi}.}}} & {(470)}\end{matrix}$Taking account of G′(x_(k))=(−1^(k)), the Equation (461) gives rise to$\begin{matrix}{{{S_{k}(x)} = \frac{( {- 1} )^{k}\quad\sin\quad\frac{\pi}{\Delta}\quad x}{\frac{\pi}{\Delta}\quad( {x - {k\quad\Delta}} )}}\quad} & {(471)} \\{= \frac{\sin\quad\frac{\pi}{\Delta}\quad( {x - x_{k}} )}{\frac{\pi}{\Delta}\quad( {x - x_{k}} )}} & {(472)}\end{matrix}$Since the derivation is independent of the regularizer ω, it followsthat our Shannon-Gabor-wavelet-DAF can be regarded as a special case ofour Lagrange DAF.

EXAMPLE 3 Dirichlet (Periodic) Sampling

Both Lagrange-sampling ans Shannon-sampling theorems hod for bandlimited functions on the entire real axis R. On might wonder whathappens for a band limited periodic finction. It is reasonable to expectsignificant reduction in the number of sampling pints required becauseofthe periodicity. This is indeed the case. Star [61] has proved that ifa function f(x) satisfies the Dirichlet boundary condition, is periodicin T and band-limited to the highest (radial frequency 2πM/T, then itcan be completely reconstructed from a finite (2M+1) set of discretesampling (grid) points [61] $\begin{matrix}{{f(x)} = {\sum\limits_{k = {- M}}^{M}\quad{{f( x_{k} )}\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{( {{2M} + 1} ){\sin\lbrack {\frac{\pi}{T}( {x - x_{k}} )} \rbrack}}}}} & (473)\end{matrix}$where Δ=T/(2M+1) is the sampling interval (grid spacing) and thex_(k)=kΔ are the sampling points. Using our standard arguments about thecontradiction between the band-limited and the physical world [62,33],and invoking the regularization procedure discussed in earlier sections,we then construct the following approximate sampling formula$\begin{matrix}{{{f(x)} \approx {\sum\limits_{k = {- W}}^{W}\quad{{f( x_{k} )}\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{( {{2M} + 1} ){\sin\lbrack {\frac{\pi}{T}( {x - x_{k}} )} \rbrack}}{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\sigma^{2}}}}}},} & (474)\end{matrix}$where W, the computational bandwidth, is smaller than M. The sampling(rid) pints {x_(k)} are distributed around the point of interest, x.Obviously when σ→∞, we recover the exact sampling theorem Equation(473). It is also interestin to examine the limit of M with a fixed Δ:$\begin{matrix}{{\lim\limits_{Marrow\infty}{\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{( {{2M} + 1} ){\sin\lbrack {\frac{\pi}{T}( {x - x_{k}} )} \rbrack}}{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\sigma^{2}}}}} = {\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{\frac{\pi}{\Delta}( {x - x_{k}} )}{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\sigma^{2}}}}} & (475)\end{matrix}$

We call$\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{\frac{\pi}{\Delta}( {x - x_{k}} )}{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\sigma^{2}}}$the Shannon-Gabor Wavelet DAF [33] (SGWD), which is a special case ofour previous Lagrange DAF [29,62]. Both the DGWD and the SGWD aregeneralizations of the (infinite grid) sinc-DVR [49,63], X (which isalso the semi-continuous Fourier DAF on a grid [25]). The DGWD reducesto the sinc-DVR in the simultaneous limits of M→∞ (with fixed Δ) andσ→∞, and the SGWD reduces to the sinc-DVR in the limit of σ→∞:$\quad\begin{matrix}{{\lim\limits_{{Marrow\infty},{\sigmaarrow\infty}}{\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{( {{2\quad M} + 1} ){\sin\lbrack {\frac{\pi}{T}( {x - x_{k}} )} \rbrack}}{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\sigma^{2}}}}} = \quad{{\lim\limits_{\sigmaarrow\infty}{\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{\frac{\pi}{\Delta}( {x - x_{k}} )}{\mathbb{e}}^{- \frac{{({x - x_{k}})}^{2}}{2\sigma^{2}}}}} = {\frac{\sin\lbrack {\frac{\pi}{\Delta}( {x - x_{k}} )} \rbrack}{\frac{\pi}{\Delta}( {x - x_{k}} )}.}}} & (476)\end{matrix}$In this sense, our DGW and SGW can be regarded as a DAF-windowedsinc-DVR or a regularized sinc-DVR. Due to the Gabor window, theregularized sinc-DVR matrix is banded. This endows the regularizedsinc-DVRs with great potential for applications in large scalecomputations.

Stark [61] has demonstrated that the Dirichlet sampling theorem canactually be derived from Shannon's sampling theorem by appropriatelyimposing the periodic boundary condition. Therefore, all three samplingtheorems are inter-related and both the Shannon sampling theorem and theDirichlet sampling theorem are special cases of the Lagrange samplingtheorem.

EXAMPLE 4 Sampling Theorems in Polar Coordinates

It is not obvious how to generate a sampling theorem in noncartesiancoordinates. In polar coordinates, the Fourier-Bessel series plays therole of Fourier series expansion in cartesian coordinates. The notion ofa band limited function is still important with band-limited referringto the Fourier-Bessel series expansion of the function which vanishesoutside a circle of radius γ. For an isotropic function f(τ) bandedlimited to γ, Camo [64] showed that $\begin{matrix}{{{f(\tau)} = {\sum\limits_{k = 1}^{\infty}{{f( \alpha_{ik} )}{S_{ik}(\tau)}}}},{and}} & (477) \\{{S_{ik}(\tau)} = \frac{2\quad\alpha_{ik}{J_{i}( {\tau\quad\gamma} )}}{\gamma\quad{J_{i + 1}( {\alpha_{ik}\gamma} )}( {\alpha_{ik}^{2} - \tau^{2}} )}} & (478)\end{matrix}$where J_(i)(·) is the ith order cylinder Bessel function of the firstkind and α_(ik)/γ is the kth zero of J_(i)(·). This result was extendedto a real function f(τ,θ) whose ith Hankel transform has compactsupport: $\begin{matrix}{{{f( {\tau,\theta} )} = {\sum\limits_{n = {- M}}^{M}{\sum\limits_{k = 1}^{\infty}{{f( {\alpha_{ik},\frac{2\quad\pi\quad n}{{2\quad M} + 1}} )}{S_{ik}(\tau)}{\Theta_{n}(\theta)}}}}},{and}} & (479) \\{{\Theta_{n}(\theta)} = {\frac{\sin\frac{{2\quad M} + 1}{2}( {\theta - \frac{2\quad\pi\quad n}{{2\quad M} + 1}} )}{\lbrack {( {{2\quad M} + 1} )\quad\sin\frac{1}{2}( {\theta - \frac{2\quad\pi\quad n}{{2\quad M} + 1}} )} \rbrack}.}} & (480)\end{matrix}$This form is useful for problems with circular boundary conditions. Byour regularization argument, w propose a corresponding polar coordinateDAF as $\begin{matrix}{{\delta_{i,M}( {{\tau - \alpha_{ik}}, {\theta - \theta_{n}} \middle| \rho_{ik} ,\sigma} )} = {\delta_{i}( {r - \alpha_{ik}} \middle| { \rho_{ik} ){\delta_{M}( {\theta - \theta_{n}} \middle| \sigma )}} }} & (481) \\{= {\frac{2\quad\alpha_{ik}{J_{i}( {\tau\quad\gamma} )}}{\gamma\quad{J_{i + 1}( {\alpha_{ik}\gamma} )}\quad( {\alpha_{ik}^{2} - \tau^{2}} )}{\mathbb{e}}^{- \frac{{({\tau - \alpha_{ik}})}^{2}}{2\quad\rho_{ik}^{2}}}\frac{\sin\lbrack {\frac{\pi}{\Delta\quad\theta}( {\theta - \theta_{n}} )} \rbrack}{( {{2\quad M} + 1} ){\sin\lbrack {{\frac{\pi}{\Delta\quad\theta}\theta} - \frac{\theta_{n}}{{2\quad M} + 1}} \rbrack}}{\mathbb{e}}^{- \frac{{({\theta - \theta_{n}})}^{2}}{2\quad\sigma^{2}}}}} & (482)\end{matrix}$This is obviously simply a product of two DAFs. It is noted that becauseof an irregular distribution of α_(ik), _(ik) is not constant and itsappropriate values need to be specified.

DISTRIBUTED APPROXIMATING FUNCTIONAL WAVELETS

Efficient and general procedures have been prescribed in previously forconstructing “delta sequence generated wavelets” from the various deltasequences descried in previously. These methods are applicable forgenerating wavelets from the various DAFs described in the last sectionbecause DAFs are, at least form the point view of functional analysis, aspecial subclass of delta sequences. We shall call the resultingwavelets “DAF-wavelets”, a terminology introduced in Ref. [32], wherevarious DAF-wavelets were created for the first time by taking thedifference of two DAFs. It is clear that the various methods describedin the last section enable one to create as many DAFs as desired.Moreover, the methods prescribed in previously will enable one togenerate infinitely many DAF wavelets from each DAF. Therefore, wediscuss only briefly a few typical DAF-wavelets in this section.However, the discussion in this section is not a simple repetition ofSection III because DAFs are multiparameter generalizations of the usualdelta sequences. Being a subclass (Schwartz-class) and a multiparametergeneralization of delta sequences facilitates conversion of tlhe DAFsinto a wide variety of DAF-wavelets.

DAF-Wavelets Generated by Differential Pairs

A general express for various DAF wavelets produced by our waveletgenerators G^(m) is $\begin{matrix}{{{\Psi_{DAF}( { x \middle| m ,\alpha,\beta,\cdots}\quad )} = {{( {{x\frac{\partial^{m}}{\partial x^{m}}} + {m\quad\frac{\partial^{m - 1}}{\partial x^{m - 1}}}} )\quad{\delta_{DAF}( { x \middle| \alpha ,\beta,\cdots} )}\quad m} = 1}},\quad 2,\quad{\cdots\quad.}} & (483)\end{matrix}$The DAF δ_(DAF)(x|m,α,β, . . . ) can be any DAF discussed in Section V.The computation is straightforward for all DAFs providing that m is nottoo large. In the case of Hermite DAFs, Equation (401), we have$\begin{matrix}{= {\frac{1}{\sigma}{\exp( {- \frac{x^{2}}{2\sigma^{2}}} )}{\sum\limits_{n = 0}^{M/2}\quad{( {- 1} )^{n + m}\frac{1}{\sqrt{2\pi}2^{{2n} + 1}{n!}}{\quad\quad\lbrack {{H_{{2n} + m + 1}( \frac{x}{\sqrt{2}\sigma} )} + {4{{nH}_{{2n} + m - 1}( \frac{x}{\sqrt{2}\sigma} )}}} \rbrack}}}}} & (484)\end{matrix}$where m, M/2=0,1,2, . . . . Here, some simple properties of Hermitefunctions have been used to simplify the results. The Mexican hatwavelet,${\frac{1}{\sqrt{2\pi}a}( {1 - \frac{x^{2}}{a^{2}}} ){\mathbb{e}}^{- \frac{x^{2}}{2\sigma^{2}}}},$and the Mexican superhat wavelet$\frac{- 2}{\sqrt{2\pi}a}( {\frac{x^{4}}{a^{4}} - \frac{6x^{2}}{a^{2}} + 3} ){\mathbb{e}}^{- \frac{x^{2}}{2b^{2}}}$are given by ψ_(0,1)(x|σ) and ψ_(0,3)(x|σ) respectively. In general, theseries of Gauss-delta-sequence-generated wavelets, Equation (396), aregiven as a special case of ψ_(0,m)(x|σ), m=0,1,2, . . . .

Since all of our nonorthogonal DAFs have the structure Tw, Equation(483) can be written also as $\begin{matrix}{{\Psi_{DAF}( { x \middle| m ,\alpha,\beta,\cdots}\quad )} = {{x{\sum\limits_{t = 0}^{m}\quad{\frac{m!}{{t!}{( {m - t} )!}}T^{(t)}w^{({m - t})}}}} + {m{\sum\limits_{t = 0}^{m - 1}\quad{\frac{( {m - 1} )!}{{t!}{( {m - t - 1} )!}}T^{(t)}w^{({m - t - 1})}}}}}} & (485)\end{matrix}$for nonorthogonal DAF wavelets. This form may be useful for theimplementation of Equation (483).

The moments M^(k) of DAF-wavelets generated using our wavelet generatorsG^(m) can be calculated as $\begin{matrix}\begin{matrix}{M_{m}^{k} = {\int{x^{k}{\Psi_{DAF}( {{x❘m},\alpha,\beta,\ldots}\quad )}{\mathbb{d}x}}}} \\{= {\int{{x^{k}( {{x\quad\frac{\partial^{m}}{\partial x^{m}}} + {m\quad\frac{\partial^{m - 1}}{\partial x^{m - 1}}}} )}\quad{\delta_{DAF}( {{x❘\alpha},\beta,\ldots}\quad )}{\mathbb{d}x}}}} \\{= \{ \begin{matrix}0 & {{k + 1} < m} \\{( {- 1} )^{m}\quad\frac{k!}{( {k - m} )!}{\int{x^{k - m + 1}{\delta_{DAF}( {{x❘\alpha},\beta,\ldots}\quad )}{\mathbb{d}x}}}} & {{{k + 1} \geq m},\quad{k - m + {1\quad{even}}}} \\0 & {{{k + 1} \geq m},\quad{k - m + {1\quad{odd}}}}\end{matrix} }\end{matrix} & (486)\end{matrix}$This expression is modified by a constant if the DAF-wavelets arenormalized in L²(R).DAF Wavelets Generated by Difference Pairs

A second class of DAF-wavelets is generated, in general, byψ_(DAF)(x|α,β, . . . ; α ^(′), β^(′), . . . )=δ_(DAF)(x|α,β, . . .)−δ_(DAF))x|α ^(′), β^(′), . . . )  (487)where at least one comparable pair of parameters, say α, α′, aredifferent from each other.

For the Hermite DAF, we have $\begin{matrix}{{\Psi_{HDAF}( {{x❘M},{\sigma\quad;M^{\prime}},\sigma} )} = {\frac{1}{\sigma}\quad{\exp( \frac{- x^{2}}{2\sigma^{2}} )}{\sum\limits_{n = {M^{\prime}/2}}^{M/2}{( {- \frac{1}{4}} )^{n}\frac{1}{\sqrt{2\quad\pi}{n!}}\quad{H_{2n}( \frac{x}{\sqrt{2}\sigma} )}}}}} & (488)\end{matrix}$The Mexican hat wavelet is obviously a special case of this generalexpression specifically ψ_(HDAF)(x|2,σ; 0,σ).

It is straightforward to generate wavelets from the Hemite DAFs of theFejér type, Equations (409) and (410). Some simple examples regardingour Dirichlet-Gabor-wavelet-packet DAF and Shannon-Gabor wavelet DAF aregiven in Refs. [32] and [33] respectively. Since DAFs already have beenfound to be extremely powerful for a variety of numerical applications,we expect that DAF-wavelets will play an important role in all of thoseareas where wavelet techniques are applicable. This is currently underinvestigation.

CONCLUSIONS

The general connection between wavelets and delta sequences (thesequences of functions which converge to the delta distribution) hasbeen spelled out in some detail.

Qualitatively, delta sequences are father wavelets (scale functions). Ifa delta sequence is an orthogonal system, it is found to span thewavelet subspace V₀ in a multiresolution analysis. Various deltasequences arising in mathematical, physical and engineering applicationsare reviewed.

A set of wavelet generators is constructed for converting deltasequences into mother wavelets. These generators are connected with aninfinite dimensional Lie algebra which has an extremely simple algebraicstructure and includes the algebra of translation and dilationoperations as an invariant subalgebra. The corresponding Lie groupprovides the basis for a mathematical description of wavelets, which ismore general than the usual translation and dilation group. A new set oforthogonal wavelets is found in the case of Dirichlet's continuous deltasequence. The well-known-Mexican hat wavelet has been shown to be aspecial case of a variety of the Hermite wavelets, and has been derivedby two distinct approaches.

The general connection between wavelet bases and frames and conventionalL²(a,b) polynomial bases was briefly discussed. Essentially, the fatherwavelet corresponds to the lowest order polynomial and all higher orderpolynomials are related to the mother wavelets, provided that thepolynomials are orthogonal with respect to some weight.

Distributed approximating functionals (DAFs) were defined asmulitparameter delta sequences of the Dirichlet type, constructed usingSchwartz-class functions. DAFs were classified as orthogonal andnonorthogonal. The former are constructed by orthogonal basis expansionof the delta distribution, and the latter are constructed by the methodof regularization. Both orthogonal and nonorthogonal DAFs are frames.The construction of orthogonal DAFs is briefly reviewed and more detailscan be found in Ref. [28]. The construction on nonorthogonal DAFs isdescribed in terms of Fourier space regularization. A general andefficient procedure for generating interpolating DAFs is presented. Theconnection between the DAF approach and mathematical sampling theory isdiscussed in detail. Various examples are given to illustrate ourapproaches. Clearly, there are infinitely many more DAFs which can beeasily constructed using our approach. For example, the well knownformula II_(j)cos(2^(−j)x) can be used to generate a DAF. It is notpossible in this paper to enumerate all the various possibilities.

A method of creating arbitrarily smoothed and arbitrarily shaped windowfunctions is briefly discussed, based on regularization. Smooth lowpass, high pass, band pass and band stop filters are constructed asspecial cases. The desired degree of smoothness is attainable by usingan appropriate (S or C^(m)) regularizing function.

A variety of DAF-wavelets (wavelets generated by using DAFs) isconstructed by using either our wavelet generators or the differencemethod. The Hermite DAF is used to illustrate our approach because inthat case, analytical forms can be obtained easily. The Mexican hatwavelet is identified as a special case of the Hermite-DAF wavelets. Weexpect that various DAF wavelets will play an important role in a widevariety of numerical applications.

All references (articles and patents) referenced or cited in thisdisclosure are incorporated herein by reference. Although the inventionhas been disclosed with reference to its preferred embodiments, fromreading this description those of skill in the art may appreciatechanges and modification that may be made which do not depart from thescope and spirit of the invention as described above and claimedhereafter.

1. A method for data padding and noise filtering implemented on adigital processing device, comprising the steps of: a. defining a totaldata set as the collection of all known and unknown data, where theunknown values may be interspersed among the known values, orconcentrated in a region adjacent to the known values, or a combinationof the two; b. placing the unknown values of the data to be obtained bya padding procedure so that the total data set contains only equallyspace data—the data has a constant sampling interval and the total dataset represents a function; c. calculating an appropriate differencebetween the true data and an approximation to the data for all known andunknown data values; d. minimizing the difference with respect to theunknown data values by iteration or using the calculus of variations toobtain algebraic equations for the unknown values or solving thealgebraic equation for the unknown data values.
 2. The method of claim1, further comprising the step of: e. removing a noise from the totaldata set using a non-interpolating, well-tempered distributedapproximating functional (NIDAF)-low-band-pass filter to form anapproximation to the total data set.
 3. The method of claim 1, whereinthe data is a time-sequence of multi-dimensional data.
 4. The method ofclaim 2, wherein time-sequence is a 2-dimensional sampling of an imageat a fixed time interval.
 5. The method of claim 1, wherein the data is1-dimensional data.
 6. A method for data padding and noise filteringimplemented on a digital processing device, comprising the steps of: a.defining a total data set as the collection of all known and unknowndata, where the unknown values may be interspersed among the knownvalues, or concentrated in a region adjacent to the known values, or acombination of the two; b. placing the unknown values of the data to beobtained by a padding procedure so that the total data set contains onlyequally space data—the data has a constant sampling interval and thetotal data set represents a function; c. calculating an appropriatedifference between the true data of the total data set and a DAFapproximation to the data for all known and unknown data values; and d.minimizing the difference with respect to the unknown data values byiteration or by using the calculus of variations to obtain algebraicequations for the unknown values or solving the algebraic equation forthe unknown data values.
 7. The method of claim 6, further comprising:b. when the known input data set comprises a finite number of sequentialdata values having an evenly spaced distribution, adding a gap of afinite number of unknown data having the same spacing or sampling rateand some other functional dependence or tail function; c. calculating anappropriate difference between the true data (known and unknown,including gap points and tail function points) and a DAF approximationto the data values including values at gap points and the unknowncoefficients in the tail function or sum of basis functions and choosingDAF parameters so that the largest predicted or estimated data values donot exceed the largest known input values and the lowest estimated orpredicted data values are not less than the lowest known input values;and d. minimizing this difference with respect to all unknown values andcoefficients by iteration or by using the calculus of variations toobtain algebraic equations for the unknowns or solving the algebraicequations for the unknowns.
 8. The method of claim 6, furthercomprising: b. when the known data set comprises a finite number ofevenly spaced, sequential data points, adding a gap with a finite numberof data point having the same spacing or sampling rate and adding arepetition of the finite number of evenly spaced, sequential data pointsto give a periodic extension of the known input data, where a signassociated with the periodic extension is the same or opposite of theinput dat and where the same sign periodic extension yields a symmetricextension of known input data and the opposite sign periodic extensionyields an anti-symmetric extension of known input data; c. calculatingan appropriate difference between the true data (known and unknown,including gap points) and a DAF approximation to this data and choosingDAF parameters so the largest estimated or predicted data do not exceedthe largest known input data and the lowest estimated or predicted dataare not less than the lowest known input data; and d. minimizing thisdifference with respect to the unknown data values by iteration or usingthe calculus of variations to obtain algebraic equations for the unknownvalues or solving the algebraic equations for the unknown gap datavalues.
 9. The method of claim 2, further comprising: b. when the knowninput data set comprises a finite number of evenly spaced (or sampled),but sequential data points, adding interspersed unknown data points toconstruct an evenly spaced or sampled data set to form an augmentsevenly spaced sequential data set, adding a gap with unknown data havingthe same spacing or sampling rate and then repeating the augmentedevenly spaced sequential data set after the gap to form a periodic,evenly spaced extension of the known data where the extension have thesame or opposite sign of the augmented data set where a same signextension yields a symmetric periodic extended data set and an oppositesign yields an anti-symmetric periodic extended data set; c. calculatingan appropriate difference between the true data (known and unknown,including gap and interspersed unknown data) and a DAF approximation tothe total data set and choosing DAF parameters so the largest estimatedor predicted data do not exceed the largest known input data, and sothat the lowest estimated or predicted data are not less than the lowestknown input data; d. minimizing this difference with respect to theunknown data values and gap and interspersed unknown data by iterationor using the calculus of variations to obtain algebraic equations forthe unknown values or solving the algebraic equations for the unknowns.10. A method for image processing implemented on a digital processingdevice, comprising the steps of: associating data of an image with agroup of points; associating a measure of disorder parameter with thedata of the image; padding the data and periodically extending it as inclaim 4 or 5, using a DAF approximation to the data with the DAFparameters chosen so that the largest estimated or predicted data valuesdo not exceed the largest known input data and the lowest estimated orpredicted data values are not less than the lowest know input data;filtering out the noise by using a well-temperedNIDAF-low-band-pass-filter to approximate the total data set andcalculating the disorder parameter as a function of the DAF parameters Mand σ|Δ. The change in the disorder parameter as a function of σ|Δ (fora given M) will be used as the indicator for the optimum well-temperedNIDAF-low-band-pass-filter; if desired, one can also use theperiodically extended and filtered total data set to carry out furtherstandard Fourier filtering.
 11. The method of claim 10, furthercomprising: categorizing the images and patterns using the finaldisorder parameter; images or patterns with the same disorder parameterwill be assumed to be similar; a catalogue of such images or patterns,and their corresponding disorder parameters, will be used to construct acomparison and categorization of surface roughness, as well as otherproperties that generate images and patterns; similar catalogues of suchimages and/or patterns of stressed materials will be used to identifyincipient structural flaws or weakness; similar catalogues of fracturedor on-going fracturing materials will be used to identify microscopicfractures or structural flaws; similar catalogues of patterns associatedwith any kind of defect will be constructed and used for predictivediagnostics.
 12. A method for data padding and noise filteringimplemented on a digital processing device, comprising the steps of: a.defining a total data set as the collection of all known and unknowndata, where the unknown values may be interspersed among the knownvalues, or concentrated in a region adjacent to the known values, or acombination of the two; b. placing the unknown values of the data to beobtained by a padding procedure so that the total data set contains onlyequally space data—the data has a constant sampling interval and thetotal data set represents a function; c. calculating an appropriatedifference between the true data of the total data set and a DAFapproximation to the data for all known and unknown data values; d.solving the algebraic equation for the unknown data values.
 13. Themethod of claim 12, further comprising the step of: e. removing a noisefrom the total data set using a non-interpolating, well-tempereddistributed approximating functional (NIDAF)-low-band-pass filter toform an approximation to the total data set.